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1  Introduction  and  Background 

One  of  the  most  significant  contemporary  trends  in  the  geospatial  sciences 
is  the  use  of  map-based  crowdsourcing  for  collecting,  confirming,  editing, 
and  displaying  geospatial  data,  referred  to  by  various  practitioners  and 
observers  as  crowdsourced  geospatial  data  (CGD),  geocrowdsourcing,  or 
volunteered  geographic  information  (VGI).  Sui  et  al.  (2013, 1)1  describe 
this  phenomenon  as  part  of  a  “profound  transformation  on  how 
geographic  data,  information,  and  knowledge  are  produced  and 
circulated,”  an  important  part  of  the  current  “exaflood  of  digital  data 
growth.”2 3 4 5  McCartney  et  al.  (2015)3  note  that  this  phenomenon  is 
permeating  the  most  basic  geographic  data  production  strategies  of  the 
United  States  Geological  Survey,  the  most  prolific  civilian  mapping  agency 
in  the  United  States.  Liu  and  Palen  trace  the  phenomenon  through  a  wide 
range  of  emergency  response  applications  in  the  government,  non-profit, 
and  media  sectors  (2010),  noting  the  influence  of  citizen-led 
geocrowdsourcing  on  professional  practice.  They  advocate  for  an  emerging 
environment  of  “interoperability  between  professional  and  participatory 
forms  of  geotechnology”.  4.5 

Going  back  at  least  eight  years  to  a  time  when  geocrowdsourcing  was  just 
emerging,  Dr.  Michael  F.  Goodchild  published  a  series  of  seminal  articles 
outlining  several  significant  benefits  associated  with  this  general 
approach;  namely,  the  local  geographic  expertise  of  the  contributors,  who 
are  more  familiar  with  the  local  features  being  mapped;  the  speed  with 
which  information  can  be  collected  and  mapped;  and  finally,  the  greatly 
reduced  costs  associated  with  what  is  typically  a  very  expensive  activity 
(2007,  2009).6 7’?  Researchers  at  George  Mason  University  (GMU)  have 


1  Daniel  Sui,  Sarah  Elwood,  and  Michael  F.  Goodchild,  eds.,  Crowdsourcing  Geographic  Knowledge 
Volunteered  Geographic  Information  (VGI)  in  Theory  and  Practice.  (New  York,  NY:  Springer,  2013). 

2  Ibid. 

3  Elizabeth  A.  McCartney  et  al.,  “Crowdsourcing  The  National  Map,”  Cartography  and  Geographic 
Information  Science  42,  no.  supl  (2015):  54-57. 

4  Ibid. 

5  S.  B  Liu  and  L.  Palen,  “The  New  Cartographers:  Crisis  Map  Mashups  and  the  Emergence  of 
Neogeographic  Practice,”  Cartography  and  Geographic  Information  Science  37,  no.  1  (2010):  69-90. 

6  Michael  F.  Goodchild,  “Citizens  as  Sensors:  The  World  of  Volunteered  Geography,”  GeoJournal  69,  no. 
4  (December  2007):  211-21. 

7  Michael  F.  Goodchild,  “NeoGeography  and  the  Nature  of  Geographic  Expertise,”  Journal  of  Location 
Based  Services  3,  no.  2  (June  2009):  82-96,  doi:10.1080/17489720902950374. 
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explored  the  topic  of  crowdsourced  geospatial  data  extensively,  through 
the  support  of  the  United  States  Army  Corps  of  Engineer’s  Geospatial 
Research  Laboratory.  The  results  of  this  work  have  been  published  in  a 
series  of  technical  reports,  conference  proceedings,  and  journal  articles 
documenting  this  work  from  its  inception  and  to  the  present  day  (Rice  et 
al.  2011,  2012a,  2012b,  2013a  2013b,  2014,  Paez  2014,  Qin  et  al.  (2015a, 
2015b)  Rice  2015,  and  Rice  et  al.  2015)8’9’10>11>12>13>14.i5,i6, 17,18  The  research 
work  from  these  publications  has  been  oriented  toward  exploring  the 
emerging  phenomena  (Rice  et  al.  2012b),  data  production  techniques 
(2012b,  2013b),  accuracy  assessment  methods  (2012b,  2013b,  2014), 
fitness-for-use  evaluations  (2012b),  exemplar  applications  and  projects 
(2012b,  2014),  and  novel  implementations  (2012b,  2014).  These  research 
topics  address  the  typical  early  questions  associated  with  a  nascent 
technological  transformation,  where  scientists,  academics,  and 


8  Matthew  T.  Rice  et  al.,  “Integrating  User-Contributed  Geospatial  Data  with  Assistive  Geotechnology 
Using  a  Localized  Gazetteer,”  in  Advances  in  Cartography  and  GIScience.  Volume  1,  ed.  Anne  Ruas, 
Lecture  Notes  in  Geoinformation  and  Cartography  (Springer  Berlin  Heidelberg,  2011),  279-91, 
http://dx.doi.org/10.1007/978-3-642-19143-5_16. 

9  Matthew  T.  Rice  et  al.,  “Supporting  Accessibility  for  Blind  and  Vision-Impaired  People  With  a  Localized 
Gazetteer  and  Open  Source  Geotechnology,”  Transactions  in  GIS  16,  no.  2  (April  2012):  177-90, 
doi:  10. 1111/j. 1467-9671.2012. 01318.x. 

10  Matthew  T.  Rice  et  al.,  “Crowdsourced  Geospatial  Data:  A  Report  on  the  Emerging  Phenomena  of 
Crowdsourced  and  User-Generated  Geospatial  Data,”  Annual  (Fairfax,  VA:  George  Mason  University, 
November  29,  2012),  http://www.dtic.mil/dtic/tr/fulltext/u2/a576607.pdf. 

11  Matthew  T.  Rice  et  al.,  “Crowdsourcing  Techniques  for  Augmenting  Traditional  Accessibility  Maps  with 
Transitory  Obstacle  Information,”  Cartography  and  Geographic  Information  Science  40,  no.  3  (June 
2013):  210-19,  doi:10.1080/15230406.2013.799737. 

12  Matthew  T.  Rice  et  al.,  “Crowdsourcing  to  Support  Navigation  for  the  Disabled:  A  Report  on  the 
Motivations,  Design,  Creation  and  Assessment  of  a  Testbed  Environment  for  Accessibility,”  US  Army 
Corps  of  Engineers,  Engineer  Research  and  Development  Center,  US  Army  Topographic  Engineering 
Center  Technical  Report,  Data  Level  Enterprise  Tools  Workgroup  (Fairfax,  VA:  George  Mason  University, 
September  2013), 

http://oai.dtic.  mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA588474. 

13  Matthew  T.  Rice  et  al.,  “Quality  Assessment  and  Accessibility  Applications  of  Crowdsourced  Geospatial 
Data:  A  Report  on  the  Development  and  Extension  of  the  George  Mason  University  Geocrowdsourcing 
Testbed,”  Annual  (Fairfax,  VA:  George  Mason  University,  September  23,  2014). 

14  Fabiana  I.  Paez,  “Recruitment,  Training,  and  Social  Dynamics  in  Geo-Crowdsourcing  for  Accessibility” 
(Master  of  Science,  George  Mason  University,  2014). 

15  Han  Qin  et  al.,  “Geocrowdsourcing  and  Accessibility  for  Dynamic  Environments,”  GeoJournal,  no. 
10.1007/sl0708-015-9659-  x  (2015):  1-18. 

16  Han  Qin  et  al.,  “Obstacle  Characterization  in  a  Geocrowdsourced  Accessibility  System,”  ISPRS  Annals 
of  Photogram metry,  Remote  Sensing  and  Spatial  Information  Sciences  1  (2015):  179-85. 

17  Rebecca  M.  Rice,  “Validating  VGI  Data  Quality  in  Local  Crowdsourced  Accessibility  Mapping 
Applications:  A  George  Mason  University  Case  Study”  (Master’s  of  Science  Thesis,  George  Mason 
University,  2015). 

18  Rebecca  M.  Rice  et  al.,  “Position  Validation  in  Crowdsourced  Accessibility  Mapping,”  Cartographical 
The  International  Journal  for  Geographic  Information  and  Geovisualization  50,  no.  4  (n.d.):  (in  press). 
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practitioners  focus  on  defining  the  terminology,  placing  boundaries  on  the 
field,  and  exploring  the  early  implementations  and  activity.  The  current 
research,  conducted  in  2014  and  2015,  asks:  “What  can  be  done  in  real 
world  settings  with  geocrowdsourced  data?”,  “What  are  the  practical 
limitations  of  the  quality  assessment  processes?”  and  “Can 
geocrowdsourced  data  be  used  in  real  dynamic,  field-based  settings?”  This 
report  addresses  these  questions,  which  focus  on  practical  quality  control, 
moderation  strategies,  and  dynamic  real-world  uses  of  geocrowdsourced 
data.  As  a  vehicle  to  explore  these  issues,  we  present  the  George  Mason 
University  Geocrowdsourcing  Testbed  (GMU-GcT),  which  was  developed 
to  explore  these  questions  and  many  more.  The  GMU-GcT  is  a  descendent 
of  earlier  mapping  systems  developed  to  help  blind,  visually-impaired,  and 
mobility-impaired  individuals  navigate  through  unfamiliar  urban  areas 
(Loomis  et  al.  2005,  Golledge  et  al.  2005  and  2006,  and  Rice  et  al. 
2005)19>20’21’22.  It  is  also  an  outgrowth  of  earlier  work  on  emerging 
technological  issues  associated  with  distributed  geographic  information 
services  and  information  sharing  communities  (Goodchild  et  al.  2005)23. 
The  context  for  the  GMU-GcT  and  the  purposes  that  it  serves  are  briefly 
explained  in  the  following  section  as  an  introduction  to  this  report.  While 
supplying  the  necessary  geoaccessibility  functionality,  an  underlying  value 
of  the  GMU-GcT  is  what  it  can  tell  us  about  the  dynamics  of 
geocrowdsourcing.  Where  possible,  these  dynamics  and  lessons  learned 
will  be  emphasized  throughout  this  report. 

The  GMU  Geocrowdsourcing  Testbed 

In  many  geospatial  settings,  the  field-based  collection  of  geospatial  data  is 
a  critical  task.  Within  dynamic  environments,  these  data  collection 
activities  are  more  challenging  due  to  the  rapid  updates  required  to 
maintain  currency  and  validity.  GMU,  for  example,  produces  a  Physical 


19  Jack  M.  Loomis  et  al.,  “Personal  Guidance  System  for  People  with  Visual  Impairment:  A  Comparison  of 
Spatial  Displays  for  Route  Guidance,”  Journal  of  Visual  Impairment  &  Blindness  99,  no.  4  (2005):  219. 

20  Reginald  G.  Golledge,  Matthew  Rice,  and  Daniel  Jacobson,  “A  Commentary  on  the  Use  of  Touch  for 
Accessing  On-Screen  Spatial  Representations:  The  Process  of  Experiencing  Haptic  Maps  and 
Graphics,”  The  Professional  Geographer  57,  no.  3  (August  2005):  339-49,  doi:10.1111/j.0033- 
0124.2005.00482.x. 

21  Reginald  G.  Golledge,  Matthew  T.  Rice,  and  R.  Daniel  Jacobson,  “Multimodal  Interfaces  for 
Representing  and  Accessing  Geospatial  Information,”  in  Frontiers  of  Geographic  Information 
Technology  ( Springer,  2006),  181-208. 

22  Matt  Rice  et  al.,  “Design  Considerations  for  Haptic  and  Auditory  Map  Interfaces,”  Cartography  and 
Geographic  Information  Science  32,  no.  4  (2005):  381-91. 

23  Michael  F.  Goodchild  et  al.,  “Report  of  the  NCGIA  Specialist  Meeting  on  Spatial  Webs”  (Santa  Barbara, 
CA:  NCGIA,  April  2005). 
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Accessibility  Map  each  August,  and  publishes  the  map  online  in  an 
attempt  to  highlight  accessible  pathways  and  areas  of  significant 
construction  activity  on  what  is  a  rapidly  changing  campus  environment 
(Figure  l).  Provided  in  static  PDF  format,  this  map  is  updated  once  a  year, 
which  makes  it  out-of-date  virtually  days  after  it  is  released.  Certain 
features,  such  as  walkways  that  meet  the  Americans  with  Disabilities  Act 
(ADA)  standard  design  criteria  for  running  slope,  are  drawn  with  a  green 
line  symbol  and  it  is  these  features  that  have  some  longevity.  A  major 
problem,  however,  emerges  when  these  features  are  made  inaccessible  by 
construction  fencing,  temporary  barricades,  and  other  transient  activity. 


Figure  1.  GMU's  Physical  Accessibility  Map24 

For  the  250-300  blind,  visually-impaired,  and  mobility-impaired  students, 
faculty,  and  staff,  changes  to  the  GMU  campus  walkways  can  present  great 
difficulty  from  the  perspective  of  navigation  and  wayfinding,  which  involve 
the  careful  planning  and  repetitive  use  of  pre-selected,  accessible 
navigation  corridors.  When  transient  obstacles  appear  on  navigation 
pathways  (Figure  2),  these  individuals  are  required  to  improvise  through 
backtracking  and  by  using  alternative  pathways. 


24  http://eagle.gmu.edu/map/pdfs/fairfax  access.pdf  [accessed,  August  10,  2015] 
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Figure  2.  Transient  Navigation  Obstacle 

Based  on  the  seminal  work  of  Loomis  et  al.  (2005,  Figure  3)2s,  Marston  et 
al.  (2006)26,  and  Golledge  et  al.  (2006)2?;  Rice  et  al.  (2011,  20i3b)28>29 
presented  the  conceptual  design  of  a  geocrowdsourcing  system  for 
collecting  transient  obstacle  information  to  assist  blind,  visually-impaired, 
and  mobility-impaired  individuals  navigate  through  unfamiliar 
environments.  The  resulting  GMU  Geocrowdsourcing  Testbed  (GMU-GcT) 
has  been  presented  and  demonstrated  in  several  previous  publications, 
notably  Rice  et  al.  (2013a,  2013b,  and  20i4).3°  .31 .32  The  purpose  of  the 
system  is  to  build  a  layer  of  quality-checked  and  validated  obstacle  data 
with  known  positional  and  temporal  characteristics  that  can  be 
disseminated  to  the  public  through  our  website,  distributed  as  a  data 
source  via  KML,  and  used  in  dynamic  routing  and  obstacle  avoidance 


25  Loomis  et  al.,  “Personal  Guidance  System  for  People  with  Visual  Impairment:  A  Comparison  of  Spatial 
Displays  for  Route  Guidance.” 

26  James  R.  Marston  etal.,  “Evaluation  of  Spatial  Displays  for  Navigation  Without  Sight,”  ACM  Trans. 
Appl.  Percept.  3,  no.  2  (April  2006):  110-24,  doi:10.1145/1141897.1141900. 

27  Golledge,  Rice,  and  Jacobson,  “Multimodal  Interfaces  for  Representing  and  Accessing  Geospatial 
Information.” 

28  Rice  et  al.,  “Integrating  User-Contributed  Geospatial  Data  with  Assistive  Geotechnology  Using  a 
Localized  Gazetteer.” 

29  Rice  et  al.,  “Crowdsourcing  to  Support  Navigation  for  the  Disabled:  A  Report  on  the  Motivations, 
Design,  Creation  and  Assessment  of  a  Testbed  Environment  for  Accessibility.” 

30  Rice  et  al.,  “Crowdsourcing  Techniques  for  Augmenting  Traditional  Accessibility  Maps  with  Transitory 
Obstacle  Information.” 

31  Rice  et  al.,  “Crowdsourcing  to  Support  Navigation  for  the  Disabled:  A  Report  on  the  Motivations, 
Design,  Creation  and  Assessment  of  a  Testbed  Environment  for  Accessibility.” 

32  Rice  et  al.,  “Quality  Assessment  and  Accessibility  Applications  of  Crowdsourced  Geospatial  Data:  A 
Report  on  the  Development  and  Extension  of  the  George  Mason  University  Geocrowdsourcing 
Testbed.” 
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scenarios.  As  discussed  in  Liu  and  Pelan  (2010)33,  the  future  of 
crowdsourcing  systems  such  as  ours,  will  likely  be  as  a  hybrid  combination 
of  professional,  authoritative  elements  and  citizen-based  neo-geographic 
elements. 


Figure  3.  UCSB  Personal  Guidance  System,  circa  2003 

Chapter  2  of  this  report  provides  an  overview  of  mobile-device  capabilities 
for  geocrowdsourcing  and  specifically,  constraints  associated  with  mobile 
device  global  positioning  systems  (GPS).  Chapter  3  provides  a  study  of  the 
social  moderation  process  in  geocrowdsourcing  and  what  it  can  provide  for 
the  GMU-GcT.  In  Chapter  4  we  look  at  the  dynamic  engagement  of 
obstacle  data  with  the  mobile,  field-based  incarnations  of  the  GMU-GcT 
and  general  positioning  capabilities  of  GPS-enabled  mobile  devices. 
Chapter  5  explores  an  extension  of  the  GMU-GcT  in  the  area  of  routing 
analysis.  The  final  chapter  of  this  report  summarizes  selected  research 
themes  for  this  project  and  addresses  what  are  viewed  as  important  future 
issues  in  geocrowdsourcing.  The  combined  retrospective  and  future  look  at 
geocrowdsourcing  is  drawn  from  our  past  technical  reports  and  the 
research  work  presented  here,  as  well  as  topics  that  we  suggest  will  be 
important  in  the  upcoming  years. 


33  Liu  and  Palen,  “The  New  Cartographers.” 
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2  Mobile  Devices  and  Geocrowdsourcing 

In  his  research  publications  George  Mason  University  (GMU)  Associate 
Professor  Dieter  Pfoser  describes  an  explosion  of  user  generated  content 
(UGC)  available  over  the  Internet  (Pfoser  2011,  Crooks  et  al.  2014)34,35.  He 
describes  contemporary  UGC  as  staggering  in  size  and  growing  larger  at  a 
rapid  pace.  His  observations  echo  Sui  et  al.’s  2013  warning  of  an 
impending  “exaflood”  of  digital  information.  Both  authors  suggest  that 
digital  geospatial  content  forms  an  important  element  in  this  tremendous 
growth,  and  a  reason  for  this  growth  is  the  widespread  availability  of 
location-aware  mobile  devices  used  to  collect  data,  photographs, 
observations,  and  information.  Pfoser  describes  this  geospatially-oriented 
UGC  as  a  rich  mixture  of  quantitative  and  qualitative  geospatial 
information,  ranging  from  georeferenced  data  collected  by  device  GPS  to 
text-based  travel  narratives,  facilitated  by  the  information  collected  and 
stored  by  location-aware  smart  phones. 

This  chapter  looks  at  the  primary  mobile  tool  associated  with  this  growing 
body  of  geospatially-oriented  UGC:  the  GPS-enabled  smart  phone.  We 
explore  the  use  of  mobile  devices  for  many  geocrowdsourcing  activities, 
including  several  current  applications  from  a  variety  of  different  domains, 
where  the  mobile  device  and  its  positioning  capabilities  facilitate  the 
central  functions  of  the  application.  We  also  present  a  review  of  mobile 
phone-based  GPS  capabilities,  and  a  study  by  project  collaborators  on  the 
positioning  constraints  of  mobile  phone  GPS  used  in  the  field. 

GPS-enabled  Dynamic  Geocrowdsourcing  Applications 

One  of  the  most  significant  drivers  of  GPS-based  data  collection  and  GPS- 
embedded  device  production  is  athletics  and  recreation.  The  tracking  of 
athletes  and  measures  of  athletic  performance  are  of  high  interest  to 
scientists  and  trainers,  who  use  GPS  tracking  for  speed  and  distance 
measurements  in  tennis  and  other  court-constrained  sports  (Duffield  et  al. 
20io)36.  Applications  in  this  area  include  GPS-enabled  geosocial  and 


34  Dieter  Pfoser,  “On  User-Generated  Geocontent,”  in  Advances  in  Spatial  and  Temporal  Databases 
(Springer,  2011),  458-61. 

35  Andrew  Crooks  et  al.,  “Crowdsourcing  Urban  Form  and  Function,”  International  Journal  of 
Geographical  Information  Science,  no.  ahead-of-print  (2014):  1-22. 

36  Rob  Duffield  et  al.,  “Accuracy  and  Reliability  of  GPS  Devices  for  Measurement  of  Movement  Patterns 
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geocrowdsourcing  applications  such  as  Strava  (Figure  4)  which  track  and 
map  the  running  events  of  end-users  and  facilitates  social  interaction 
through  a  geosocial  community.  Strava  uses  mobile  GPS  to  calculate 
distance,  speed,  rate,  elevation  gain,  and  if  equipped  with  GPS  waypoints 
for  a  pre-defined  running  course,  can  calculate  distance  deviations  from 
the  defined  course.  Strava  uses  a  map-based  and  social-community  based 
engagement  strategy  to  encourage  connections  between  athletes  in  local 
areas.  The  map-based  tracking  functionality  using  device  GPS  represents  a 
key  capability  in  the  mobile  tools  designed  for  the  GMU-GcT,  while  the 
geosocial  functionality  in  this  app  represents  an  important  recruitment 
and  engagement  aspect  of  geocrowdsourcing,  addressed  by  Paez  (2014) 
and  Rice  et  al.  (2014). 


in  Confined  Spaces  for  Court-Based  Sports,”  Journal  of  Science  and  Medicine  in  Sport  13,  no.  5 
(2010):  523-25. 
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Figure  4.  GPS-enabled  geosocial  athletics  and  run-tracking  applications37 


Geocaching  is  the  modern  incarnation  of  a  treasure  hunt  game,  where 
GPS-enabled  mobile  devices  are  used  to  locate  hiding  places,  called 
‘caches’  and  track  items  left  in  the  cache  and  carried  to  other  caches  by 
participants.  The  most  popular  mobile  geocaching  application,  produced 
by  Groundspeak,  Inc.  and  distributed  through  the  Apple  App  Store  as 
“Geocaching,”  contains  map-based  and  GPS-based  searching  capabilities, 
with  orientation  and  distance  displays,  including  an  uncertainty  estimate 
for  GPS,  popup  reminders  about  device  GPS  capabilities  and  the  current 
search  radius  (Figure  5).  There  are  more  than  2  million  geocaches  hidden 


37  https://www.strava.com/  [accessed  August  23,  2015] 
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in  184  countries  world-wide,  and  an  estimated  user  base  of  3-4  million. 38>39 
The  spatial  searching  functionality  in  the  Geocaching  App  and  focus  on 
distance  between  user  and  object  using  mobile  device  GPS  is  similar  to 
functionality  in  mobile  GMU-GcT  tools  discussed  in  this  report. 
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Figure  5.  Geocaching  GPS-enabled  mobile  application 

Many  contemporary  geocrowdsourcing  applications  are  transportation- 
related,  which  reflects  the  view  expressed  by  Pfoser  and  others,  that  one  of 
the  most  significant  emerging  trends  in  current  geocrowdsourcing  is  the 
generation  of  tracking  data  used  for  transportation  analysis  and  related 
transportation  and  logistics  activities.  The  combination  of  dynamic  data 
collection  and  analysis  is  at  the  foundation  of  the  mobile  GMU-GcT 
activities  profiled  in  this  report,  where  we  focus  on  the  interaction  of  end- 
users  and  moderators  with  location-aware  mobile  devices  that  both 
contribute  and  receive  geocrowdsourced  data  from  the  field. 

The  local  public  transportation  network  serving  the  City  of  Fairfax  and 
GMU  includes  signage  with  QR  codes  that  when  scanned  with  a  mobile 
device,  opens  a  dynamic  bus  tracking  service  that  provides  distance  and 
time  estimates  between  a  user’s  location  and  the  bus’s  location  (Figure  6). 
This  tracking  data  is  dynamically  updated  through  a  web  application  and  a 
GPS  bus  tracking  system.  On  the  back  end,  the  next  bus  service  can 
analyze  the  patterns  associated  with  service  requests  generated  from 


38  http://www.geocaching.com/blog/2013/02/celebratin-two-million-geocaches-list-by-country/ 

[accessed  August  23,  2015] 

39  http://forums.groundspeak.com/GC/index.php?showtopic=241632  [accessed  August  23,  2015] 
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specific  QR  codes,  and  provide  information  to  public  transportation 
providers  about  location-specific  queries,  which  could  be  used  to  improve 
infrastructure  or  focus  service  on  locations  with  high  QR  code  scanning 
activity. 


Figure  6.  Sign-mounted  QR  code  for  City  of  Fairfax  CUE  bus  tracking  through  the  NextBus 

Service40 

The  GPS  Alarm  Clock  application  (Figure  7)  is  a  novel  application  for 
travelers  and  public  transit  riders,  similar  to  the  accessibility  application 
developed  by  Barbeau  et  al.  (2010), 41  to  provide  notification  of  proximity 
to  a  planned  stop.  This  is  targeted  toward  inattentive  or  sleeping 
passengers,  as  well  as  disabled  travelers  who  need  advanced  warning  of  an 
approaching  stop.  The  application  uses  the  built-in  GPS  device  uncertainty 
estimates,  provided  as  a  blue  bubble  around  the  present  GPS  location,  as 
well  as  a  distance  threshold  for  triggering  warnings.  The  functionality  in 
this  application  is  directly  relevant  to  our  interest  in  providing  mobile 
obstacle  notifications  through  the  GMU-GcT. 


40  https://www.nextbus.com  [accessed  August  23,  2015] 

41  Sean  J.  Barbeau  et  al.,  “Travel  Assistance  Device:  Utilising  Global  Positioning  System-Enabled  Mobile 
Phones  to  Aid  Transit  Riders  with  Special  Needs,"  Intelligent  Transport  Systems ,  IET  4,  no.  1  (2010): 
12-23. 
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Figure  7.  Location  and  scale-sensitive  GPS  Alarm  for  commuters 

Finally,  Yik  Yak  is  a  geosocial  application  that  has  received  a  great  deal  of 
attention  over  the  last  two  years,  due  to  the  anonymous  nature  of  the 
interactions  among  end-users.  The  concept  of  the  geosocial  application  is  a 
geographically-defined  user  community,  which  includes  all  users  within  a 
specific  radius  of  the  end-user.  This  user  community  dynamically  changes 
based  on  the  location  of  the  device.  As  a  useful  dynamic  element,  the 
application  shuts  down  when  it  detects  that  it  is  located  near  to  or  inside  of 
the  footprint  for  a  public  school  facility,  other  than  colleges  and 
universities,  where  it  operates  freely  (Figure  8).  This  geofencing 
functionality  was  implemented  to  curb  the  anonymous  trolling  and  anti¬ 
social  elements  that  became  widespread  among  younger  users.  This  novel 
method  for  controlling  mobile  device  interaction  using  geographic  location 
highlights  the  capabilities  for  device-based  GPS  to  be  a  part  of 
fundamental  interaction  in  geocrowdsourcing  and  geosocial  media.  Figure 
37  shows  the  application  being  shut  down  due  to  proximity  to  Fairfax  High 
School. 
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Figure  8.  Yik-Yak  geosocial  mobile  app  with  geofencing  capabilities42 

Mobile  Device  GPS  Accuracy  Studies 

In  a  2009  study  of  the  iPhone  3G,  Zandbergen  studied  the  horizontal  error 
patterns  for  mobile  device  GPS,  finding  a  Root  Mean  Square  Error 
(RMSE)  of  8.3  meters,  and  a  maximum  error  of  18.5  meters  (Figure  9)43. 
The  Garmin  GPS  device  used  as  a  control  had  a  maximum  error  of  1.4m 
and  an  RMSE  of  1.0  meters. 


42  http://www.yikyakapp.com/  [accessed  August  23,  2015] 


43  Paul  A.  Zandbergen,  “Accuracy  of  iPhone  Locations:  A  Comparison  of  Assisted  GPS,  WiFi  and  Cellular 
Positioning,"  Transactions  in  GIS  13,  no.  si  (2009):  5-25. 
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Figure  9.  Horizontal  error  plot  for  Garmin  and  iPhone  GPS,  from  Zandbergen  (2009) 

An  earlier  study  by  Modsching  et  al.  (2006)44  looked  at  GPS  accuracy  for 
travelers  and  tourists  in  a  medium-sized  city.  They  found  error  rates  for 
four  different  consumer-level  GPS  receivers  to  be  between  2.5  meters  (for 
open  areas  such  as  public  squares)  and  15.4  meters  (for  urban  streets  with 
four  story  buildings  on  each  side,  Figure  10).  The  Modsching  et  al.  study 
provides  estimates  for  GPS  accuracy  in  the  type  of  public  spaces  and 
buildings  heights  encountered  by  end-users  of  the  GMU-GcT. 


44  ibid. 
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Figure  10.  GPS  accuracy  study  for  two  urban  settings,  from  Modsching  et  al.  (2006) 

GMU  Geocrowdsourcing  Testbed  Mobile  Device  GPS  Accuracy  Study 

GMU  Doctoral  student  and  research  collaborator  Rodney  Vese  has  been  an 
instrumental  part  of  the  State  of  Maryland’s  public  mapping  and  GIS 
efforts,  with  responsibility  for  studying  the  feasibility  of  citizen-based 
mapping  of  public  trails.  In  order  to  understand  how  to  conflate  trail  data 
geocrowdsourced  with  mobile  devices,  and  provide  quality  assessment  for 
position,  Rodney  has  begun  a  large  study  of  consumer-device  capabilities 
under  a  variety  of  conditions  and  circumstances.  Some  of  his  preliminary 
data  is  presented  here,  along  with  error  estimates  that  confirm  the 
estimates  of  Zandbergen  (2009)45  and  Modsching  et  al.  (2006)46. 

Vese  uses  methodology  based  on  the  Frechet  distance,  which  is  a  measure 
for  the  distance  between  two  curves,  proposed  by  Frechet  (1906)47.  In  a 
much  later  study,  Alt  et  al.  (199s)48  gave  an  algorithm  for  the  computation. 
A  popular  illustration  of  the  Frechet  distance  is  the  following:  Suppose  a 
person  is  walking  his  dog,  the  person  is  walking  on  the  one  curve  and  the 


45  ibid. 

46  Marko  Modsching,  Ronny  Kramer,  and  Klaus  ten  Hagen,  “Field  Trial  on  GPS  Accuracy  in  a  Medium 
Size  City:  The  Influence  of  Built-Up,”  in  3rd  Workshop  on  Positioning ,  Navigation  and  Communication, 
2006,  209-18. 

47  M.  Maurice  Frechet,  “Sur  Quelques  Points  Du  Calcul  Fonctionnel,”  Rendiconti  Del  Circolo  Matematico 
Di  Palermo  (1884-1940)  22,  no.  1  (1906):  1-72. 

48  Helmut  Alt  and  Michael  Godau,  “Computing  the  Frechet  Distance  between  Two  Polygonal  Curves,” 
International  Journal  of  Computational  Geometry  &  Applications  5,  no.  01n02  (1995):  75-91. 
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dog  on  the  other.  Both  are  allowed  to  control  their  speed  but  they  are  not 
allowed  to  go  backwards.  Then  the  Frechet  distance  of  the  curves  is  the 
minimal  length  of  a  leash  that  is  necessary  for  both  to  walk  the  curves 
from  beginning  to  end.  Figure  n,  from  Brakatsoulas  et  al.  (2005)49 
provides  a  visual  example  of  the  Frechet  distance  for  two  curves,  Q  and  P. 


Discrete  Frechet  Distance  of  cutves  P  and  Q:  2.1124 


Figure  11.  Frechet  Distance  for  curves  P  and  Q,  from  Brakatsoulos  et  al.  (2005) 

This  general  methodology  is  used  to  compare  the  curve  generated  by  a 
survey-grade  Trimble  GPS  with  iPhone4,  iPhones,  and  iPhone6  GPS 
curves.  The  mobile  devices  utilize  Collector  for  ArcGIS  to  automatically 
stream  device  GPS  coordinates  to  ArcGIS  Online  while  being  carried  along 
trails  or  pedestrian  pathways.  The  mobile  GPS  data  is  collected  and  stored 
using  editable  feature  service  layers  from  Esri’s  Spatial  Database  Engine 
(ArcSDE),  with  double  precision  coordinates  preserved  through  the  entire 
processing  chain.  Additional  post-processing  routines  are  used  to  check 
data  consistency  and  add  calculated  Frechet  distances  to  each  segment.  An 
excerpt  from  these  mobile  device  tracks  is  shown  in  Figure  12  and  Figure 
13  (below),  as  part  of  a  large  data  collection  transect  through  our  study 
area.  The  transect  began  in  the  center  of  the  GMU  campus,  proceeding  1.5 
miles  to  the  center  of  the  City  of  Fairfax,  and  returned  on  the  same  path, 
using  the  sidewalks. 


49  Sotiris  Brakatsoulas  etal.,  “On  Map-Matching  Vehicle  Tracking  Data,”  in  Proceedings  of  the  31st 
International  Conference  on  Very  Large  Data  Bases  (VLDB  Endowment,  2005),  853-64. 
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Figure  12.  Ground-level  view  of  mobile  device  GPS  tracks,  Fairfax,  Virginia 


Figure  13.  Overhead  view  of  mobile  device  GPS  tracks,  Fairfax,  Virginia 
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After  the  data  collection  transect  is  finished,  Frechet  distances  are 
calculated  and  stored  as  attributes  of  each  curve  segment  using  post¬ 
processing  routines  in  MATLAB.  Average  Frechet  distances  for  each 
device  are  calculated  for  30-second  time  intervals  and  stored  separately. 
Figure  14,  Figure  15,  and  Figure  16  (below)  show  average  Frechet  distances 
for  a  section  of  this  transect  near  the  north  side  of  the  GMU  campus,  with 
the  iPhone  4  have  a  Frechet  distance  of  6.42  meters,  iPhone  5  having  a 
Frechet  distance  of  5.62  meters,  and  iPhone  6  having  a  Frechet  distance  of 
4.75  meters. 


Table  1  has  a  summary  of  average  Frechet  distance  for  each  device  along  3 
component  tracks  of  the  larger  transect,  and  an  average  Frechet  distance 
for  the  total  observations  from  all  tracks.  In  this  case,  the  average  Frechet 
distance  for  the  iPhone  4  is  10.51  meters,  the  iPhone  5  is  6.72  meters,  and 
the  iPhone  6  is  5.92  meters. 


iPhone  4 
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Figure  14.  Mobile  GPS  tracks  and  average  Frechet  distance  for  iPhone  4 
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Figure  15.  Mobile  GPS  Tracks  and  Average  Frechet  Distance,  iPhone  5 
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Figure  16.  Mobile  GPS  Tracks  and  Average  Frechet  Distance,  iPhone  6 


Table  1.  Average  Frechet  distances  for  mobile  device  GPS  (meters)  with  weighted  average. 


Average  Frechet  Distances 


Frechet 

Weighted 

Distance 

Average 

Track  1 

5.89 

iPhone  6 

Track  2 

5.11 

5.92 

Track  3 

7.94 

Track  1 

5.26 

iPhone  5 

Track  2 

6.68 

6.72 

Track  3 

10.44 

Track  1 

8.71 

iPhone4 

Track  2 

11.32 

10.51 

Track  3 

13.15 

21 


In  addition  to  average  Frechet  distances,  the  mobile  device  GPS  accuracy 
study  conducted  for  this  research  project  looked  at  the  type  of  canopy 
cover  (or  alternatively),  land  use/land  cover  as  a  factor  responsible  for 
significant  variation  in  GPS  accuracy.  The  Modsching  study  (2006)  found 
mobile  device  GPS  to  be  more  accurate  in  open  areas,  such  as  public 
squares,  and  less  accurate  near  tall  buildings.  Preliminary  work  by  Vese  in 
the  state  of  Maryland  confirms  this  general  trend.  In  the  local  study  area, 
Vese  found  mobile  device  GPS  tracks  to  have  more  variation  in  areas  with 
tall  buildings,  likely  due  to  multipath  error,  where  the  signal  received  by 
the  mobile  GPS  is  being  reflected  from  tall  structures.  Figure  17  shows  this 
dynamic,  with  all  GPS  tracks  within  6.62  meters  of  each  other  near  the 
City  of  Fairfax  City  Hall,  which  is  characterized  as  ‘Open  Canopy’.  The  area 
just  1/3  mile  north  on  University  Drive,  near  the  GMU  Commerce  building 
shows  mobile  device  tracks  spreading  to  a  width  of  23.16  meters,  where 
larger  5-6  story  buildings  predominate. 


Figure  17.  Mobile  GPS  tracks,  Open  Canopy  (A)  and  Urban  (B) 

A  thorough  analysis  of  canopy  coverage  types  and  average  Frechet 
distance  is  presented  in  Figure  18,  Figure  19,  Figure  20,  Figure  21  and 
Table  3,  from  a  collection  of  multiple  data  transects  through  the  State  of 
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Maryland.  The  average  Frechet  distance  for  areas  with  Open  Canopy  was 
7.53  meters,  Partial  Canopy  was  7.07  meters,  Heavy  Canopy  was  9.47 
meters,  and  Urban  Canopy  (including  taller  buildings)  was  14.83  meters. 
Each  of  these  average  Frechet  distances  were  calculated  across  multiple 
transects  with  thousands  of  individual  observations.  An  analysis  of 
variance  (ANOVA)  for  the  four  canopy  types  (Table  3)  shows  an  F-statistic 
of  6.02  and  an  associated  p-value  of  0.0014.  The  critical  F-statistic  value 
for  this  test  (with  3  and  49  degrees  of  freedom  and  alpha  =  0.05)  is  2.794. 
Based  on  these  numbers,  we  can  conclude  that  the  null  hypothesis, 
suggesting  no  significant  variation  between  treatment  groups  (canopy 
types),  can  be  rejected.  For  the  four  canopy  types,  at  least  one  of  them  is 
different  than  the  others. 


OPEN  CANOPY 


Figure  18.  Mobile  GPS  data  transect  with  average  Frechet  distance  (Open  Canopy) 


Frechet  Distance  (meters)  31  Frechet  Distance  (meters) 
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PARTIAL  CANOPY 


Mobile  GPS  Transect 


■  iPhone4 

■  iPhone  5 


igure  19.  Mobile  GPS  data  transect  with  average  Frechet  distance 


(Partial  Canopy) 


HEAVY  CANOPY 


Mobile  GPS  Transect 


■  iPhone4 

■  iPhone  5 


Figure  20.  Mobile  GPS  data  transect  with  average  Frechet  distance  (Heavy  Canopy) 
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URBAN  CANOPY 


Figure  21.  Mobile  GPS  data  transect  with  average  Frechet  distance  (Urban  Canopy) 


Additional  pairwise  tests  for  the  average  Frechet  distances  by  canopy  type 
(Table  3  and  Table  4),  with  the  Bonferroni  correction,  indicate  that  only 
the  Heavy  Canopy  type  is  different  than  the  Open  and  Partial  Canopy.  All 
other  pairwise  comparisons  yield  results  where  the  null  hypothesis 
(equality  of  means)  cannot  be  rejected.  It  is  important  to  note  that  the 
Urban  Canopy  type  (Figure  21,  Table  2,  Table  3,  and  Table  4)  has  a  much 
larger  variance  than  the  other  types,  and  therefore  cannot  be  distinguished 
as  different  during  the  pairwise  tests  for  means.  The  shape  of  the  graph  for 
Urban  Canopy  in  Figure  21  reflects  the  same  variation  seen  in  Figure 
17(B),  where  tall  buildings  result  in  interference,  likely  from  multipath 


error. 
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Table  2.  Single  factor  ANOVA  for  Canopy  Type 


Single  Factor  ANOVA  for  Canopy  Type 


SUMMARY 


Groups 

Count 

Sum 

Average 

Variance 

Open 

22 

165.8593 

7.539061 

6.988686 

Partial 

8 

56.63584 

7.079479 

7.269384 

Heavy 

15 

142.0906 

9.472703 

4.346086 

Urban 

8 

118.6585 

14.83231 

99.13493 

ANOVA 

Source  of  Variation 

SS 

df 

MS  F 

P-value 

F  crit 

Between  Groups 

351.206 

3 

117.0685  6.022817 

0.001414 

2.79395 

Within  Groups 

952.438 

49 

19.43751 

Total 

1303.64 

52 

Table  3.  Test  for  Equality  of  Variances  for  Average  Frechet  Distance  by  Canopy  Type.  Green 
figures  show  homoscedastic  pairs,  red  figures  show  heteroskedastic  pairs. 

Equalilty  of  Variances  (F-tests,  Observed  /  Critical) 


Open 

Partial 

Heavy 

Urban 

Open 

1.04  /  2.48 

1.61/2.38 

14.19  /  2.49 

Partial 

1.04  /  2.48 

1.67/2.76 

13.64/3.79 

Heavy 

1.61/2.38 

1.67/2.76 

22.81/2.76 

Urban 

14.19/2.49 

13.64/3.79 

22.81/2.76 

Table  4.  Pairwise  T-Tests  for  Average  Frechet  Distance  by  Canopy  Type.  Green  figures  show 
means  with  statistical  equality,  orange  figures  show  means  with  inequality. 


Pairwise  T-tests  (T-value,  P-value  |  T-critical  at  0.05) 


Open 

Partial 

Heavy 

Urban 

Open 

0.419,  0.678  |  2.04 

2.37, 0.023  |  2.03 

2.045,  0.080  |  2.364 

Partial 

0.419,  0.678  |  2.04 

2.369,  0.027  |  2.07 

2.125,  0.066  |  2.306 

Heavy 

2.37,  0.023  |  2.03 

2.369,  0.027  |  2.07 

1.505,  0.176  |  2.364 

Urban 

2.045,  0.080  |  2.364 

2.125,  0.066  |  2.306 

1.505,  0.176  |  2.364 
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For  GPS-based  mobile  device  positioning,  we  can  assume  that  positioning 
will  vary  based  on  whether  or  not  the  end-user  is  near  tall  buildings,  or 
perhaps  under  heavy  canopy.  Zandbergen  cites  a  GPS  horizontal  accuracy 
figure  of  8.3  meters  (RMSE)  with  a  maximum  of  18.5  meters.  Modsching 
et  al.  found  the  GPS  horizontal  accuracy  to  vary  between  2.5  and  15.4 
meters,  based  on  whether  the  location  was  open  or  near  tall  buildings. 
Vese’s  GPS  study  yields  confirmatory  results,  with  average  Frechet 
distances  between  7.52  and  14.8  meters  generally,  and  for  our  specific 
study  area,  between  5.92  meters  and  10.51  meters,  depending  on  the 
device  used.  The  Vese  GPS  study,  and  earlier  studies  by  Zandbergen  and 
Modsching  et  al.  suggest  that  mobile  devices  have  approximate  accuracies 
between  2.5  meters  and  15.4  meters,  likely  somewhere  between. 
Applications  with  a  need  for  higher  accuracy,  i.e.,  transportation  and 
navigation,  often  employ  additional  methods  to  snap  the  device’s  map- 
based  position  to  the  nearest  network,  an  item  explored  by  Karagiorgou,  et 
al.  (2012),  Pfoser  (2000),  Pfoser  et  al.  (2003,  2005),  and  Brakatsoulas  et 
al.  (2005)5°’  51-  52, 53, 54.  For  the  GMU-GcT,  the  dynamics  for  obstacle 
interaction  will  be  guided  by  additional  uncertainty  and  interaction 
buffering,  and  for  moderator  search,  the  mobile  GPS  accuracy  will  place 
the  moderator  well  within  the  normal  visual  identification  distances. 

The  quality  of  the  obstacle  positioning  in  the  GMU-GcT  is  based  both  on 
the  device  GPS  accuracy  (for  reports  submitted  by  end-users)  as  well  as 
map-based  positioning  and  multiple  aspects  of  moderated  position 
validation,  as  discussed  in  Chapter  3  of  this  report,  Rice  (2015)55,  and  Rice 
et  al.  (2015)56.  For  the  GMU-GcT,  we  use  information  from  GPS  accuracy 
studies  summarized  above  to  control  the  interaction  distances  between 
end-users  and  moderators  with  mobile  devices,  and  obstacles  in  our 


50  Sophia  Karagiorgou  and  Dieter  Pfoser,  “On  Vehicle  Tracking  Data-Based  Road  Network  Generation,”  in 
Proceedings  of  the  20th  International  Conference  on  Advances  in  Geographic  Information  Systems 
(ACM,  2012),  89-98. 

51  Dieter  Pfoser,  “Issues  in  the  Management  of  Moving  Point  Objects”  (Department  of  Computer  Science, 
the  Faculty  of  Engineering  and  Science,  Aalborg  University,  2000). 

52  Dieter  Pfoser  and  Christian  S.  Jensen,  “Indexing  of  Network  Constrained  Moving  Objects,”  in 
Proceedings  of  the  11th  ACM  International  Symposium  on  Advances  in  Geographic  Information 
Systems  (ACM,  2003),  25-32. 

53  Dieter  Pfoser  and  Christian  S.  Jensen,  “Trajectory  Indexing  Using  Movement  Constraints*,” 
Geolnformatica  9,  no.  2  (2005):  93-115. 

54  Brakatsoulas  et  al.,  “On  Map-Matching  Vehicle  Tracking  Data.” 

55  Rice,  “Validating  VGI  Data  Quality  in  Local  Crowdsourced  Accessibility  Mapping  Applications:  A  George 
Mason  University  Case  Study.” 

56  Rice  et  al.,  “Position  Validation  in  Crowdsourced  Accessibility  Mapping.” 
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system.  Figure  22  shows  a  conceptual  drawing  of  an  end-user  with  some 
positioning  uncertainty  (based  on  his/her  mobile  GPS  device  positioning 
error,  summarized  in  this  chapter)  traversing  a  path  with  obstacles,  each 
of  which  has  some  final  position  uncertainty  associated  with  the  quality  of 
the  social  moderation  process  (summarized  in  Chapter  3  of  this  report). 

This  chapter  suggests  that  the  position  uncertainty  for  mobile  device  users 
in  this  scenario  is  between  5.92  and  10.51  meters,  depending  on  the  device 
used  and  the  presence  of  tall  buildings  and  heavy  canopy  (Table  2).  The 
analysis  presented  in  Chapter  3  of  this  report,  and  in  Rice  (2015)  suggests 
that  the  uncertainty  in  obstacle  positioning  in  the  GMU-GcT  is  between 
2.12  meters  and  5.55  meters.  The  combination  of  the  two  uncertainty 
ranges  (shown  conceptually  as  dashed  concentric  buffers  in  Figure  22,  is 
8.04  meters  and  16.06  meters.  This  suggests  that  in  circumstances  where  a 
GMU-GcT  user  has  a  mobile  device  with  the  best  positioning  (iPhone  6, 
5.92  meters)  and  obstacles  have  been  optimally  moderated  for  position 
(2.12  meters),  the  average  uncertainty  between  positions  is  expected  to  be 
approximately  8.04  meters. 

Mobile  devices  with  less  accurate  GPS  capabilities  and  GMU-GcT  obstacles 
with  less  accurate  moderated  positions  may  lead  to  distances  of  16  meters 
or  more.  For  GMU-GcT  moderators  interacting  with  unverified  end-user 
reports  with  positional  error  (summarized  in  Rice  et  al.  2014  as  having  an 
average  positional  error  of  18.36  meters),  the  uncertainty  in  position 
between  the  moderator  (with  a  mobile  device  have  between  average 
positional  uncertainty  of  5.92  -  10.51  meters)  and  the  obstacle  interaction 
distance  would  be  24.28  -  28.87  meters.  The  interaction  distance 
formulations  discussed  above  are  summarized  in  Table  5  (A)  for 
interaction  with  moderated  obstacles,  and  Table  5  (B)  for  mobile 
interaction  with  unmoderated  reports. 
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Figure  22.  Conceptual  diagram  of  the  Interaction  between  end-user  and  obstacles  in  the 

GMU-GcT 


Table  5.  Interaction  distances  for  moderated  obstacles  and  for  field  moderation 


A).  Interaction  distance  estimates  for  moderated  GMU-GcT  obstacles 

iPhone  6 

iPhone  4 

Position  uncertainty  of  moderated  GMU-GcT  obstacles 

2.12 

5.55 

Position  uncertainty  from  Mobile  device  GPS 

5.92 

10.51 

Interaction  Distances  (meters) 

8.04 

16.06 

B).  Interaction  distance  estimates  for  GMU-GcT  field  moderation 

iPhone  6 

iPhone  4 

Position  uncertainty  of  unmoderated  reports: 

18.36 

18.36 

Position  uncertainty  from  Mobile  device  GPS: 

5.92 

10.51 

Interaction  Distances  (meters) 

24.28 

28.87 
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Mobile  Field  Moderation  in  the  GMU-GcT 

The  moderation  activities  in  the  GMU-GcT,  covered  in  detail  in  Rice  et  al. 
(2014)57,  Rice  et  al.  (2015)58.59,  and  Qin  et  al.  (2015a),  have  in  the  past 
involved  a  hybrid  desktop  and  mobile  interaction  paradigm,  where 
information  is  collected  via  mobile  device  and  entered  into  a  desktop- 
based  moderation  dashboard.  Project  researchers  have  decomposed  the 
basic  functionalities  of  this  moderation  dashboard  (presented  in  Rice  et  al. 
20 14)60  and  deployed  them  to  a  mobile  Web  Application  design  for  use  on 
a  mobile  phone.  The  benefits  of  this  moderation  paradigm  have  been 
immediate  and  clear.  The  GMU-GcT  mobile  field  moderation  tools  allow 
moderators  to  see  and  inspect  obstacles,  collect  photographs,  and 
document  their  properties  while  viewing  the  original  obstacle  reports.  This 
in-situ  moderation  activity  is  not  only  more  efficient  and  satisfying,  as 
reported  by  project  moderators,  but  is  quicker,  due  to  all  moderation  being 
done  in  the  field  rather  than  on  separate  computers  over  a  period  of  hours. 
We  provide  a  description  of  the  tool  and  the  mobile  moderation  workflow 
here. 

The  GMU-GcT  mobile  moderation  portal  uses  the  same  PostgreSQL 
database,  PHP  code,  and  JavaScript  as  our  desktop  moderation 
dashboard,  but  uses  a  JQuery  Mobile  framework  to  develop  the  mobile 
interface  and  functionality.  Direct  connections  between  our  GMU-GcT 
native  mobile  application  and  the  mobile  moderation  portal  are  provided 
through  HTML  links,  so  that  a  moderator  can  use  the  more  responsive 
native  mobile  application  while  in  exploratory  mode  or  routing  mode,  but 
switch  to  the  web  application  for  mobile  moderation.  Figure  23  shows  two 
preliminary  obstacle  selection  screens  (a  single  screen  with  finger  scrolling 
functionality  displayed  here  as  two  images).  This  screen  is  used  by 
moderators  to  select  and  preview  reports,  and  contains  the  key  fields  for 
obstacle  reports,  including  User  ID,  Report  ID,  Location,  Obstacle 
Description,  Obstacle  Type,  and  the  contributed  images.  Experimentation 
with  the  mobile  moderation  tools  and  the  reorganization  of  the 


57  Rice  et  al.,  “Quality  Assessment  and  Accessibility  Applications  of  Crowdsourced  Geospatial  Data:  A 
Report  on  the  Development  and  Extension  of  the  George  Mason  University  Geocrowdsourcing 
Testbed.” 

58  Rice  et  al.,  “Position  Validation  in  Crowdsourced  Accessibility  Mapping.” 

59  Qin  et  al.,  “Geocrowdsourcing  and  Accessibility  for  Dynamic  Environments.” 

60  Rice  et  al.,  “Quality  Assessment  and  Accessibility  Applications  of  Crowdsourced  Geospatial  Data:  A 
Report  on  the  Development  and  Extension  of  the  George  Mason  University  Geocrowdsourcing 
Testbed.” 
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moderation  workflow  underscored  the  importance  of  having  the  Obstacle 
Image  and  map-based  location  as  key  elements  on  the  first  screen  as  an 
essential  part  of  the  preview.  In  our  former  moderation  paradigm,  obstacle 
position  was  moderated  several  steps  before  an  image  of  the  obstacle  was 
moderated,  but  for  mobile  moderation,  the  obstacle  image  needed  to  be 
part  of  the  first  screen.  As  we  discuss  in  the  final  chapter  of  this  report, 
obstacle  images  are  the  primary  ‘commodity’  and  future  focus  on  the 
GMU-GcT. 
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Figure  23.  Obstacle  selection  and  preview  screen  of  the  GMU-GcT  Mobile  Moderation  Portal 

After  an  obstacle  preview,  the  moderator  taps  the  ‘next’  button  at  the 
bottom  of  the  page.  At  this  point,  the  moderation  tools  switch  to  a  non¬ 
linear  structure  with  six  different  screen  areas  representing  different 
aspects  of  moderation  (Figure  24).  The  six  moderation  areas  can  be  visited 
in  sequence  or  out  of  order  with  a  single  tap. 
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Figure  24.  Main  Moderation  menu  of  the  GMU-GcT  Mobile  Moderation  Portal 

Figure  25  shows  the  location  validation  functionality,  (accessed  through 
tapping  #1,  Figure  24),  which  opens  a  screen  with  a  map-based  obstacle 
position  that  can  be  zoomed,  and  obstacle  position  updated  with  a  click- 
drag  action.  Positional  accuracy  for  the  report  (calculated  with  a 
Haversine-based  distance  calculation  between  the  reported  location  and 
the  moderated  location)  is  displayed  and  stored  in  our  PostgreSQL 
database. 
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Figure  25.  Location  validation  in  the  GMU-GcT  Mobile  Moderation  Portal 

Figure  26  contains  the  moderation  menus  for  checking  and  updating 
location  description,  obstacle  description,  obstacle  type,  and  obstacle 
impact,  which  are  accessed  by  tapping  #2,  Figure  24).  The  moderated 
values  are  stored  in  our  PostgreSQL  database. 
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Figure  26.  Obstacle  descriptions  update  in  the  GMU-GcT  Mobile  Moderation  Portal 

Image  validation  (Figure  27)  is  accomplished  through  the  tools  behind 
item  #3  from  Figure  24.  The  screen  allows  moderators  to  quickly  view  and 
replace  images  of  an  obstacle,  and  provide  an  assessment  of  image  quality 
from  our  Moderator  Rubric  (discussed  in  Rice  et  al.  2014  and  Rice 
2015)61’62. 


si  Ibid. 

62  Rice,  “Validating  VGI  Data  Quality  in  Local  Crowdsourced  Accessibility  Mapping  Applications:  A  George 
Mason  University  Case  Study." 
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Figure  27.  Image  validation  in  the  GMU-GcT  Mobile  Moderation  Portal 

The  remaining  screens  (accessed  through  tapping  #4,  5,  6  from  Figure  24) 
take  the  moderator  through  an  assessment  of  obstacle  attributes,  including 
duration,  urgency,  and  status  code  (Figure  28);  moderator  comments, 
report  quality  scoring,  and  obstacle  lifetime  extensions  (if  needed,  Figure 
29);  and  general  moderation  overview  (Figure  30).  These  items  are 
followed  by  a  confirm  /  submit  process,  where  quality  assessment 
statistics  are  displayed  and  final  moderation  submission  is  made  (Figure 
3i). 
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Figure  28.  Attributes  and  status,  GMU-GcT  Mobile  Moderation  Portal 
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Figure  29.  Moderator  comments,  scoring,  and  obstacle  longevity 
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Figure  30.  Obstacle  moderation  overview,  GMU-GcT  Mobile  Moderation  Portal 
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Figure  31.  Quality  assessment  statistics  and  moderation  overview  with  submit 


37 


The  following  chapter  presents  an  important  overview  and  study  of  the 
social  moderation  process,  which  is  at  the  heart  of  the  quality  assessment 
for  the  GMU-GcT,  and  represents  one  of  the  principle  quality  assessment 
techniques  outlined  by  Goodchild  and  Li  (2012)  for  geocrowdsourcing 
applications.^ 


63  Michael  F.  Goodchild  and  Linna  Li,  “Assuring  the  Quality  of  Volunteered  Geographic  Information,” 
Spatial  Statistics  1  (May  2012):  110-20,  doi:10.1016/j.spasta.2012.03.002. 
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3  Social  Moderation  for  Crowdsourced 
Geospatial  Data 

Quality  assessment  and  social  moderation  processes  in 
geocrowdsourcing 

Though  crowdsourced  geospatial  data  (CGD)  has  proved  to  be  useful  in 
providing  up-to-date  information,  the  quality  and  the  reliability  of  these 
datasets  are  two  areas  of  concern.  In  the  Phase  3  report  (Rice  et  al. 
20i4)64,  several  approaches  to  ensuring  geocrowdsourced  data  quality 
were  reviewed,  along  with  the  approach  for  data  quality  assessment  used 
by  the  GMU  Geocrowdsourcing  Testbed  (GMU-GcT).  Large  web  service 
platforms  such  as  Wikipedia  and  OpenStreetMap  (OSM)  rely  on  a  large 
contributor  base  to  find  and  correct  errors  (a  crowdsourced  approach), 
resulting  in  continuously  improved  database  quality.  However,  the  GMU- 
GcT  cannot  solely  rely  on  this  approach,  due  to  its  smaller  contributor 
group.  Since  the  GMU-GcT  produces  a  reasonably  small  dataset,  with  an 
average  contribution  rate  of  ten  reports  per  week,  we  apply  what 
Goodchild  and  Li  (20i2)6s  refer  to  as  the  social  approach  to 
geocrowdsourced  quality  assessment.  This  approach  relies  on  a  group  of 
team  leaders  to  act  as  moderators  in  validating  incoming  data. 

The  use  of  moderators  to  validate  data  is  necessary  when  there  is  no 
authoritative  data  source  to  authenticate  geocrowdsourced  information, 
though  it  is  uncertain  how  accurate  this  method  is.  The  GMU-GcT  utilizes 
a  team  of  moderators  who  are  thoroughly  trained  in  project  details, 
obstacle  classification,  and  precise  methods  for  moderating  reports 
including  field-checking.  The  aim  is  for  consistency  amongst  one  another 
in  order  to  establish  authority  as  the  ground-truth.  However,  there  is  the 
concern  regarding  the  extent  to  which  the  moderators  are  consistent  with 
one  another,  as  well  as  if  they  are  a  sufficient  source  for  the  ground-truth. 
This  section  will  report  on  an  analysis  of  the  consistency  and  adequacy  of 
multiple  moderators  in  assessing  the  quality  of  GMU-GcT  data  in  both 
positional  location  and  obstacle  characterization.  For  position,  we  base 


64  Rice  et  al.,  “Quality  Assessment  and  Accessibility  Applications  of  Crowdsourced  Geospatial  Data:  A 
Report  on  the  Development  and  Extension  of  the  George  Mason  University  Geocrowdsourcing 
Testbed,”  201. 

65  Goodchild  and  Li,  “Assuring  the  Quality  of  Volunteered  Geographic  Information.” 
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adequacy  on  consistency  with  previous  findings  in  other  geocrowdsourced 
data  quality  studies,  such  as  Haklay  (2010)66,  Girres  and  Touya  (2010)6?, 
and  Ramm  et  al.  (2011) 68,  who  find  that  positional  error  is  typically  within 
5-6  meters  in  OSM  studies  (as  summarized  in  Table  6).  We  also  use 
Camponovo  and  Freundschuh’s  (2014)69  research  on  prior  knowledge  of 
categorical  accuracy  of  geocrowdsourced  data.  The  data  used  in 
Camponovo  and  Freundschuh  (2014)  is  from  the  Ushahidi  Project,  a 
humanitarian  relief  project  that  took  place  during  the  2010  Haiti 
earthquake,  where  emergency  responders  were  asked  to  categorize 
incoming  messages  into  a  series  of  primary  and  secondary  categories.  The 
accuracy  rates  for  placement  into  primary  categories  were  50%  and 
subcategories  were  only  27%.  The  tasks  being  performed  by  emergency 
responders  were  difficult  and  in  some  cases,  unfamiliar,  and  the  messages 
being  categorized  involved  language  translation,  which  presents  further 
difficulty.  Nevertheless,  the  Camponovo  and  study  highlights  the  need  for 
analysis  of  uncertainties  associated  with  categorization  by  individuals 
involved  in  geocrowdsourcing.  The  GMU-GcT  involves  several 
opportunities  for  contributors  to  characterize  attributes  and  select 
categories.  Other  than  the  preliminary  study  by  Paez  (2014)  and  this  study 
by  Rice  (2015),  we  have  not  pursued  the  quality  issues  associated  with 
errors  in  categorization  to  the  same  depth  as  presented  in  the  Camponovo 
study. 


66  Mordechai  Haklay,  “How  Good  Is  Volunteered  Geographical  Information?  A  Comparative  Study  of 
OpenStreetMap  and  Ordnance  Survey  Datasets,”  Environment  and  Planning.  B ,  Planning  &  Design  37, 
no.  4  (2010):  682. 

67  Jean-Frangois  Girres  and  Guillaume  Touya,  “Quality  Assessment  of  the  French  OpenStreetMap 
Dataset,”  Transactions  in  GIS  14,  no.  4  (August  2010):  435-59,  doi:10.1111/j.l467- 

9671. 2010. 01203. x. 

68  Frederik  Ramm,  Jochen  Topf,  and  Steve  Chilton,  OpenStreetMap:  Using  and  Enhancing  the  Free  Map 
of  the  World  (UIT  Cambridge  Cambridge,  2011),  http://library.wur.nl/WebQuery/clc/1958758. 

69  Michael  E.  Camponovo  and  Scott  M.  Freundschuh,  “Assessing  Uncertainty  in  VGI  for  Emergency 
Response,”  Cartography  and  Geographic  Information  Science  41,  no.  5  (October  20,  2014):  440-55, 
doi:  10.1080/15230406.2014.950332. 
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Table  6.  Summary  of  positional  accuracy  findings  for  various  OpenStreetMap  datasets. 


Reference 

Datasets  Analyzed 

OpenStreet  Reference 

Map  Dataset 

Positional  Accuracy 

Points  Lines 

Buffer 

used 

Girres  and 

Touya  2010 

France,  2009 

Ordnance 

Survey 

6.65  m 

Hausdorff  distance  = 

13.57m 

Average  distance  = 

2.19m 

Haklay  2010 

England, 

2008 

Ordnance 

Survey 

5.83m 

3.75m  and 

5.6m 

Arsanjani  et 

al.  2013 

Germany, 

2012 

Federal  Agency 

for  Cartography 

and  Geodesy 

(BKG) 

3m,  5m, 

10m,  15m 

Ciepluch  et 

al.  2010 

Ireland,  2010 

Google  Maps 

Used  a  point  system  to  measure  number  of 

errors  for  road  types 

Ramm  et  al. 

2011 

Generalizes  that  data  contributed  by  users 

using  GPS  is  within  5m 

Moderator  Consistency  Study 

In  evaluating  moderator  consistency,  a  study  was  conducted  as  a  part  of 
Rebecca  Rice’s  Master’s  Thesis  (2015).  Ms.  Rice  designed  a  study  in  which 
datasets  of  three  project  members  who  acted  as  GMU-GcT  moderators 
were  tested  for  consistency.  First,  the  three  moderators  were  trained  in  the 
use  of  moderator  tools  by  Rebecca,  the  study  designer.  Each  moderator 
received  a  rubric,  carefully  created  by  the  project  leader,  providing 
detailed  instructions  on  how  to  adequately  position  reports  to  reflect  the 
actual  obstacle  location  as  well  as  how  to  correctly  classify  obstacles.  Then 
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the  study  designer  reports  a  series  of  obstacles  to  separate  customized 
versions  of  the  GMU-GcT  website,  enabling  simultaneous  blind 
moderation  by  all  three  moderators.  As  soon  as  new  reports  were  added  to 
the  separate  databases  and  appeared  on  each  moderator’s  web  map,  each 
moderator  was  asked  to  field-check,  confirm,  and  quality  assess  the  report. 
Due  to  the  transient  nature  of  the  obstacles  being  reported,  moderators 
were  asked  to  field-check  and  moderate  reports  on  the  same  day  the 
reports  are  were  contributed.  Once  each  of  the  moderators  completed  their 
set  of  reports  for  the  day,  we  created  a  “ground-truth”  version  of  each 
reported  obstacle  by  making  a  joint  decision  with  the  two  study  designers 
and  project  leader  through  careful  moderation  of  the  report,  based  on  the 
rubric  created  by  the  project  leader.  This  ground-truth  version  acted  as  the 
basis  for  comparison  in  evaluating  moderator  accuracy.  The  workflow  of 
the  study  is  outlined  in  Figure  32. 
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Figure  32.  Framework  outlining  the  process  for  evaluating  moderator  consistency 


Data  Collection 

Over  the  course  of  three  weeks  in  April  2015,  a  set  of  33  reports  with 
varying  quality  were  contributed  to  the  test  GMU-GcT  database  being  used 
for  the  study  by  Rebecca,  project  leader  and  study  designer.  Between  four 
and  seven  obstacles  were  reported  per  day,  twice  a  week,  and  then  copied 
to  the  three  moderator  PostgreSQL  databases.  The  moderators  were  then 
notified  that  there  were  reports  to  field-check,  moderate,  and  validate. 
Figure  33  highlights  the  study  area  as  well  as  the  location  of  the  reported 
obstacles. 
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Figure  33.  Obstacles  report  to  the  GMU-GcT  for  the  moderator  consistency  study  (33  total) 
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Evaluating  Positional  Accuracy 

Because  the  GMU-GcT  uses  its  data  for  obstacle-avoidance  in  routing, 
positional  accuracy  is  critical,  as  demonstrated  by  Figure  34.  Once 
moderated,  obstacles  are  expected  to  intersect  with  the  GMU-GcT’s 
pedestrian  network.  To  evaluate  positional  accuracy,  the  position  of  each 
moderated  report  was  compared  to  the  ground-truth  report  for  each 
obstacle.  The  “near”  tool  in  ArcMap  was  used  to  calculate  the  distance 
from  one  feature  to  its  nearest  feature  based  on  the  projected  coordinate 
system  of  the  data,  resulting  in  the  distance  of  each  moderated  report  from 
the  actual  obstacle.  These  distances  are  analyzed  to  determine  the  overall 
positional  error  and  gain  insight  into  moderator  accuracy  in  positioning 
reports. 


Figure  34.  Obstacle  avoidance  in  routing  for  the  GMU-GcT.  Obstacles  are  buffered  to  facilitate 
interaction  through  routing  and  mobile  exploration.  An  unconfirmed  report  may  not  intersect 

with  the  pedestrian  network. 
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Evaluating  categorical  consistency  for  obstacle  type 

Because  the  GMU-GcT  is  meant  to  be  useful  for  end-users  with  varying 
accessibility  needs,  it  is  important  that  obstacles  are  correctly  categorized. 
The  classification  system  for  obstacle  type  is  outlined  extensively  in  the 
rubric  given  to  the  moderators.  The  GMU-GcT  uses  six  main  categories  to 
classify  obstacles:  sidewalk  obstruction,  construction  detour,  poor  surface 
condition,  entrance/exit  problem,  crowd/event,  and  other.  Moderators  can 
select  one  or  multiple  obstacle  types  during  the  validation  process.  The 
reports  evaluated  by  the  moderators  in  this  study  contained  varying  data 
quality,  meaning  that  some  reports  did  not  need  obstacle  type  changes, 
while  other  reports  needed  considerable  attention.  After  the  data 
collection,  all  moderator  databases  were  analyzed,  and  each  obstacle 
report  was  given  a  score  based  on  consistency  with  ground  truth.  Reports 
received  a  “2”  for  a  complete  match,  a  “1”  for  a  partial  match,  and  a  “o”  for 
no  match.  Averaging  all  of  the  reports  scores  for  each  moderator  provided 
a  final  score  on  overall  categorical  consistency  for  the  obstacle  type 
category. 

Results 

Thirty-three  obstacles  were  reported  during  the  course  of  this  study, 
resulting  in  a  total  of  132  data  entries  for  quality  analysis.  The  study  area, 
campus  sections  1-4,  containing  the  ground-truth  location  of  the  obstacles 
along  with  the  position  chosen  by  each  of  the  moderators,  is  shown  in 
Figure  35,  Figure  36,  Figure  37,  and  Figure  38,  while  Table  7  summarizes 
the  positional  measures  for  the  33  collected  reports.  The  average 
positional  error  amongst  the  three  moderators  is  5.55  meters,  with  a 
standard  deviation  of  2.86  meters.  Out  of  the  33  obstacle  reports,  54.5%  of 
reports  were  positioned  within  three  meters  of  the  obstacle’s  actual 
location,  which  is  well  within  the  expected  distance  as  laid  out  by  previous 
research.  69.7%  of  the  reports  were  positioned  within  6  meters  of  the 
obstacle’s  actual  location,  which  is  in  congruence  with  similar  findings 
regarding  positional  accuracy  of  VGI.  Figure  39  is  a  representation  of  two 
reports  that  were  positioned  very  well  by  the  moderators,  which  is  ideal  in 
the  realm  of  geocrowdsourced  data  quality.  An  analysis  of  variance 
(ANOVA)  for  comparing  the  mean  positional  accuracy  figures  for  the  three 
moderators  (Table  8)  indicates  that  there  were  no  significant  differences 
between  them  (F-statistic  1.5735,  p-value  0.2162).  An  analysis  of  the  type 
of  obstacle  (point,  areal)  indicates  that  the  mean  positional  accuracy  for 
point  features  (2.12  meters)  is  significantly  different  from  the  mean 
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positional  accuracy  for  areal  features  (12.54  meters).  This  comparison  is 
based  on  a  Student’s  T-statistic  with  heteroskedastic  variance  and  an 
unequal  sample  size,  results  in  a  t-  value  of  4.817,  a  p-value  of  0.00002, 
with  38  degrees  of  freedom  (Table  9). 


Figure  35.  Obstacle  reported  to  the  GMU-GcT  for  the  moderator  consistency  study,  including 
the  obstacle's  actual  report  location  and  the  locations  chosen  by  moderators.  South  GMU 

Campus 
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Figure  36.  Obstacle  reported  to  the  GMU-GcT  GcT  for  the  moderator  consistency  study, 
including  the  obstacle’s  actual  location  and  the  locations  chosen  by  moderators.  North  GMU 

Campus 
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Figure  37.  Obstacle  reported  to  the  GMU-GcT  for  the  moderator  consistency  study,  including 
the  obstacle’s  actual  location  and  the  locations  chosen  by  moderators.  Old  Town  Fairfax 
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Figure  38.  Obstacle  reported  to  the  GMU-GcT  for  the  moderator  consistency  study,  including 
the  obstacle's  actual  location  and  the  locations  chosen  by  moderators.  North  Fairfax. 
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Table  7.  Positional  error  for  each  report,  based  on  the  location  determined  by  each 
moderator.  Positional  error  is  calculated  as  the  difference  between  the  moderator  position 
and  the  confirmed  ground  truth  position. 


Positional  Errors  of  Moderated  Reports 

Report 

Modi 

Mod2 

Mod3 

AVGPE 

STDEV 

Areal 

Feature? 

report_0001(mod) 

2.25 

0.65 

1.14 

1.35 

0.82 

report_0002(mod) 

3.59 

3.69 

0.16 

2.48 

2.01 

report_0003(mod) 

2.74 

2.80 

3.38 

2.97 

0.35 

report_000365(mod) 

15.79 

20.21 

13.55 

16.52 

3.39 

Y 

report_000366(mod) 

11.35 

4.07 

4.27 

6.56 

4.15 

Y 

report_000367(mod) 

0.66 

1.10 

0.42 

0.73 

0.35 

report_000368(mod) 

1.07 

0.48 

0.94 

0.83 

0.31 

report_000369(mod) 

17.36 

8.43 

3.63 

9.81 

6.97 

Y 

report_0004(mod) 

2.98 

2.24 

3.18 

2.80 

0.50 

report_0005(mod) 

1.77 

4.72 

2.20 

2.90 

1.59 

report_0007(mod) 

27.48 

35.71 

4.45 

22.55 

16.20 

Y 

report_0008(mod) 

5.17 

5.61 

5.12 

5.30 

0.27 

report_0009(mod) 

14.93 

6.12 

5.32 

8.79 

5.33 

Y 

report_00010(mod) 

7.97 

2.03 

2.57 

4.19 

3.28 

Y 

report_00011(mod) 

3.98 

2.06 

1.77 

2.60 

1.20 

report_00012(mod) 

3.00 

3.38 

2.39 

2.92 

0.50 

report_00013(mod) 

1.00 

2.22 

3.79 

2.34 

1.40 

report_00014(mod) 

3.52 

2.60 

1.26 

2.46 

1.13 

report_00016(mod) 

8.28 

1.56 

5.44 

5.09 

3.37 

Y 

report_00017(mod) 

4.09 

19.70 

16.59 

13.46 

8.26 

Y 

report_00018(mod) 

2.22 

26.95 

1.53 

10.23 

14.48 

Y 

report_00019(mod) 

4.26 

2.64 

16.46 

7.79 

7.55 

Y 

report_00020(mod) 

5.71 

55.14 

4.73 

21.86 

28.83 

Y 

report_00021(mod) 

9.09 

0.37 

1.89 

3.78 

4.66 

report_00022(mod) 

4.10 

4.83 

1.86 

3.59 

1.55 

report_00023(mod) 

0.72 

0.76 

0.57 

0.69 

0.10 

report_00024(mod) 

0.93 

5.05 

0.76 

2.25 

2.43 

report_00025(mod) 

4.17 

0.25 

0.25 

1.56 

2.27 

report_00026(mod) 

25.87 

5.79 

3.70 

11.79 

12.24 

Y 

report_00027(mod) 

0.36 

0.59 

0.51 

0.49 

0.12 

report_00029(mod) 

0.27 

0.13 

0.17 

0.19 

0.07 

report_00030(mod) 

1.07 

0.37 

1.07 

0.84 

0.41 

report 00031(mod) 

0.60 

1.83 

2.07 

1.50 

0.79 

Average  Positional  Error 

6.01 

7.09 

3.55 

5.55 

2.86 

Average  Positional  Error 
(Areal  Features  Removed) 

2.53 

2.18 

1.66 

2.12 

4.15 
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Table  8.  Analysis  of  Variance  for  Moderator  Positional  Accuracy 


Analysis  of  Variance  (One-Way) 


Summary 

Groups 

Sample  size 

Sum 

Mean 

Variance 

modi 

33 

198.35048 

6.01062 

48.85601 

mod2 

33 

234.07044 

7.09304 

141.05155 

mod3 

33 

117.13823 

3.54964 

17.54808 

ANOVA 

Source  of  Variation 

SS 

df 

MS 

F 

p-level 

Fcrit 

Between  Groups 

217.62105 

2 

108.81052 

1.5735 

21.26176% 

3.09119 

Within  Groups 

6,638.58048 

96 

69.15188 

Total 

6,856.20152 

98 

Table  9.  T-Test  for  Obstacle  Type  (point,  areal) 


T-Test:  Positional  Accuracy  for  Obstacle  Type 


Descriptive  Statistics _ 

_ VAR _ Sample  size _ Mean _ Standard  Deviation _ Variance 

point  63  2.12136  1.78096  3.17183 

areal  38  12.54855  13.27182  176.14116 


t-test  assuming  unequal  variances  (heteroscedastic) 

Degrees  of  Freedom  38 

Hypothesized  Mean  Difference  0. 

Pooled  Variance  67.81 694 

Test  Statistics  4.81707 


Two-tailed  distribution _ 

p-ievel_ 0.00002  Critical  Value  (5%) 2.02439 

One-tailed  distribution _ 

p-level_ 0.00001  Critical  Value  { 5 %) 1.68595 


^  . 

0.94^# 1  07 

• 

' 

0.48 

2.24  f"f 
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\  A  </us> 

2.98  \W 

8 

> 

3.18 

\ 

report_0004 

report_000368 

Average  Positional  Error:  2.80  m 

Average  Positional  Error:  0.83  m 

Standard  Deviation:  0.50  m 

Standard  Deviation:  0.31  m 

Figure  39.  A  representation  of  the  positional  errors  associated  with  two  obstacle  reports.  The 
black  circles  represent  the  obstacle  location  chosen  by  the  moderators,  while  the  asterisk 
represents  the  obstacle’s  actual  location,  as  determined  by  the  project  managers. 
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Six  of  the  ninety-three  moderator  obstacle  reports  had  positional  errors  of 
greater  than  20  meters.  This  can  mainly  be  attributed  to  some  obstacles 
not  being  adequately  represented  by  a  point  location,  which  is  the  type  of 
feature  used  to  represent  obstacles  in  our  system,  rather  than  a  line  or 
polygon  representation.  This  presents  an  issue  for  obstacles  that  are  very 
large,  as  demonstrated  by  Figure  40.  In  many  cases,  the  obstacles  reported 
to  the  GMU-GcT  consisted  of  a  large  construction  area  or  a  large  segment 
of  sidewalk  that  was  closed,  which  would  be  best  represented  by  a  polygon 
feature.  The  moderators  for  this  study  were  instructed  to  assign  the 
position  of  the  obstacle  report  directly  in  the  center  of  the  obstacle,  but  the 
resulting  variances  in  spatial  perception  associated  with  georeferencing 
became  a  factor  in  selecting  the  accurate  position  of  the  obstacle. 
Report_ooc>7  (Figure  41)  is  an  example  of  such  scenario,  which  explains 
the  positional  error  of  22.55  meters.  Because  of  this  discrepancy  Table  7 
has  a  column  indicating  whether  an  obstacle  was  an  areal  feature,  and  the 
overall  positional  error  was  recalculated  with  areal  features  removed,  with 
an  average  positional  error  of  non-areal  features  being  2.12  m. 


Figure  40.  A  representation  of  the  limitations  presented  by  assigning  position  as  a  point 
location  rather  than  assigning  position  to  encompass  a  larger  area,  as  a  line  or  polygon. 
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report_0007 

Average  Positional  Error:  22.55  m 
Standard  Deviation:  16.20  m 


Figure  41.  Demonstration  of  a  difficult  report  to  position  with  a  point  due  to  its  areal  nature, 
resulting  in  a  high  positional  error. 


The  difference  in  average  positional  errors  between  the  point  and  areal 
features  underscores  the  inadequacy  in  simple  point-based  representation. 
This  problem  has  been  partially  rectified  with  polygon-based  obstacle 
footprints,  but  this  feature  is  not  commonly  used  by  the  public.  Polygon- 
based  obstacle  footprints  are  enabled  for  regular  moderation  in  the  GMU- 
GcT,  however,  they  were  not  implemented  in  this  study  due  to  its  potential 
for  creating  complications  with  the  moderating  process.  The  regular  GMU- 
GcT  moderation  process  will  continue  to  use  polygon  geometries  to 
represent  obstacles  that  are  larger  and  inadequately  represented  by  a 
point. 

Another  issue  that  caused  a  larger  positioning  error  amongst  moderators 
is  the  limitation  of  georeferencing  based  off  of  Google  Maps  orthoimagery, 
as  opposed  to  collecting  GPS  coordinates  of  hazards  when  field  checking. 
The  orthoimagery  provided  by  Google  is  not  leaf-off,  which  causes  issues 
for  positioning  in  some  cases,  such  as  report_ooo365  (Figure  42). 
Report_ooo365  consisted  of  a  wooden  walkway  that  routed  the  user 
around  a  pathway  that  required  a  detour  due  to  construction,  and  is  not 
reflected  in  the  most  recent  orthoimagery  or  in  Google  Maps,  making  the 
obstacle  even  more  difficult  to  locate  on  a  map. 
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Figure  42.  An  example  of  a  poorly  positioned  report,  due  to  the  limitations  present  by  using 
Google  orthoimagery  to  georeference  an  obstacle’s  position  rather  than  validating  position  by 
GPS  or  through  some  other  method. 


In  evaluating  categorical  consistency  for  obstacle  type,  moderators 
remained  mostly  consistent.  After  calculating  a  matching  score  for  the 
category  designations  in  each  report,  where  o=no  match,  impartial  match, 
and  2=exact  match,  a  total  matching  score  was  calculated  for  each 
moderator  for  all  reports.  The  total  matching  score  for  moderators  was 
1.79  for  moderator  1, 1.76  for  moderator  2,  and  1.70  for  moderator  3 
(Figure  43).  The  overall  average  matching  score  for  all  moderators  and  all 
reports  was  1.75.  This  shows  that  the  three  moderators  were  mostly  in 
agreement,  and  have  an  overall  76.8%  exact  match  rate,  and  an  overall 
98.0%  match  rate  of  at  least  one  of  the  appropriate  categories.  The  match 
rate  for  each  moderator  compared  to  the  ground  truth  is  graphed  in  Figure 
43- 
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Category  Match  Rate  Detail  for  Moderators 


Moderator  l 
1.79  /  2.00 


Moderator  2 
1.76  /  2.00 


78.78% 


no  partial  exact 
match  match  match 


Moderator  3 
1.70  /  2.00 


30 


match  match  match 


Figure  43.  Overall  agreement  between  the  three  moderators  and  the  ground  truth  for 
classifying  obstacles,  with  each  moderator  represented  by  one  histogram. 


While  an  exact  match  rate  of  76.8%  is  very  good,  and  is  better  than 
categorical  accuracy  found  in  research  such  as  Camponovo  and 
Freundschuh  (2014),  there  is  still  some  explanation  necessary  for  some 
matching  errors  and  disagreement  in  obstacle  classification.  Though  the 
moderators  are  trained  and  given  a  rubric  that  declares  the  criteria  for 
each  obstacle  type,  there  is  still  some  ambiguity.  For  example,  poor  surface 
conditions  and  sidewalk  obstructions  can  easily  be  confused  if  a  poor 
surface  condition  begins  to  fully  or  partially  obstruct  a  path,  in  which  it 
would  be  a  sidewalk  obstruction  as  well.  Construction  detours  are  also 
sometimes  difficult  to  classify,  because  there  are  often  reports  for 
construction-related  hazards,  but  they  do  not  actually  culminate  in  a 
detour.  This  was  apparent  for  one  obstacle,  in  which  two  of  the  moderators 
resulted  in  “no  match”  for  correctly  classifying  obstacle  type.  The  original 
report  contained  “construction  detour,  sidewalk  obstruction,”  though  the 
obstacle  would  have  been  correctly  classified  as  a  poor  surface  condition. 
One  moderator  classified  the  obstacle  correctly,  while  another  moderator 
classified  the  obstacle  as  a  construction  detour,  and  another  as  a  sidewalk 
obstruction.  Figure  44  exemplifies  some  of  the  discrepancies  for  each 
obstacle  type,  with  the  most  discrepancies  being  associated  with  sidewalk 
obstructions  and  poor  surface  conditions. 
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report 

■  Ground 
Truth 

■Mod  1 
"Mod  2 
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Figure  44.  Total  count  of  obstacle  type  selections  per  each  moderator,  compared  to  original 

reports  and  the  ground  truth. 


This  data  quality  analysis  is  a  progressive  start  to  assessing  consistency 
amongst  moderators  and  evaluating  the  social  moderator  process.  We 
have  gained  much  insight  from  this  study  about  the  role  that  moderators 
play  in  positioning  reports,  and  the  expected  positional  accuracy,  which  is 
better  than  the  typical  positional  accuracies  for  mobile  GPS  reported  by 
authors  such  as  Zandbergen  (2009),  reviewed  in  Chapter  2  of  this  report. 

The  GMU-GcT  benefits  from  its  small  area  of  interest  and  relatively  small 
amount  of  incoming  data  in  that  the  social  approach  is  most  appropriate 
for  ensuring  data  quality.  In  order  to  assure  that  incoming  reports  are 
checked  in  a  timely  manner,  the  GMU-GcT  employs  a  team  of  moderators 
to  quality  check  reports.  The  conclusions  that  can  be  drawn  from  this 
study  are  that  moderator  positioning  falls  within  the  expected  positional 
error  of  most  crowdsourced  data  (as  outlined  by  previous  OSM  study 
findings  in  Table  7),  meaning  that  the  GMU-GcT  contains  data  that  falls 
within  the  quality  standard  provided  by  other  VGI  quality  assessment 
studies.  While  we  can  assert  that  the  data  contained  in  the  GMU-GcT  is 
quality  assured,  it  is  difficult  to  assert  that  our  data  is  “good  quality” 
without  a  set  standard  for  VGI  data  quality,  similar  to  the  National 
Standard  for  Spatial  Data  Accuracy. 
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While  moderators  are  thoroughly  trained  in  how  to  field-check  and 
validate  obstacles,  there  is  no  proven  method  for  making  them  100% 
consistent  with  one  another.  While  the  reports  are  mainly  consistent,  in 
that  their  positioning  is  within  3  meters  for  all  obstacles  (and  less  than  1 
meter  for  non-area  features),  and  in  agreement  for  almost  all  obstacle 
types,  there  is  still  some  inherent  ambivalence  in  selecting  a  location  as 
well  as  for  determining  obstacle  type.  However,  as  the  research  from 
Camponovo  and  Freundschuh  (2014)  shows,  ambivalence  maybe  an 
intrinsic  part  of  the  geocrowdsourcing  process.  In  order  to  improve  the 
consistency  and  accuracy  of  moderator’s  decisions,  it  is  critical  that  we 
ensure  they  are  trained  and  are  allowed  to  make  joint  decisions  when 
validating  incoming  data.  If  this  experiment  allowed  the  three  moderators 
to  discuss  with  one  another  the  reasoning  behind  their  report  positioning 
and  categorical  selections,  they  would  likely  be  more  consistent  and  more 
congruent  with  the  ground  truth.  While  the  only  categorical  assessment 
analyzed  in  depth  for  this  study  was  obstacle  type,  the  many  other  areas  of 
categorical  assessment  will  be  analyzed  in  future  to  gauge  moderator 
consistency.  A  preliminary  look  at  categorical  designations  in  the  GMU- 
GcT  is  reviewed  in  Paez  (2014)7°  and  Qin  et  al.  (2015b)?1. 

Multiposition  Validation  and  Moderation  in  the  GMU-GcT 

One  recent  avenue  of  research  by  this  team  is  the  combination  of  multiple 
elements  of  positioning  to  provide  additional  guidance  and  help  to  the 
GMU-GcT  moderators.  As  noted  in  this  chapter,  each  report  submitted  by 
a  contributor  includes  an  asserted  position,  as  well  as  a  moderated 
position.  The  moderated  report  also  includes  a  moderated  position,  which 
was  the  subject  of  the  study  in  this  chapter.  Other  report  elements  such  as 
the  images  and  description  text  can  produce  other  positioning  information 
that  can  be  combined  into  a  multiposition  validation  workflow,  which  is 
still  under  development.  Figure  45  shows  the  concept  of  multiple  aspects 
of  position  validation  in  the  GMU-GcT.  including  user-contributed 
position  from  smartphone  GPS  and  hand-selected  georeferencing,  as  well 
as  final  validated  positioning  built  with  moderator  positioning,  image 
geotags,  and  geoparsed  location  description.  This  section  will  look  at  the 
development  of  experimental  methods  to  use  image  geotags  associated 
with  position  and  orientation,  as  well  as  geoparsed  location  description 
text  for  multiposition  validation  in  the  GMU-GcT. 


70  Paez,  “Recruitment,  Training,  and  Social  Dynamics  in  Geo-Crowdsourcing  for  Accessibility. 

71  Qin  et  al.,  “Obstacle  Characterization  in  a  Geocrowdsourced  Accessibility  System.” 
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Moderator 

Position 


Figure  45.  Multiple  aspects  of  positioning  in  the  GMU-GcT,  and  the  position  confirmation 

process 


Image  Geotags  for  Position  and  Orientation 

Figure  46  shows  typical  GMU-GcT  report  with  an  attached  photograph, 
submitted  with  the  report.  Report  contributors  can  submit  up  to  two 
photographs,  while  additional  reports  of  the  same  obstacle  can  contribute 
additional  images.  Figure  47  shows  a  conceptual  view  of  information  that 
can  be  extracted  from  a  collection  of  reports  with  multiple  photographs  of 
the  same  obstacle.  The  figure  shows  positional  information  and 
orientation  vectors  extracted  from  the  image  headers  of  three 
photographs,  whose  origin  positions  and  intersection  points  would 
converge  on  the  reported  obstacle,  whose  position  is  being  independently 
confirmed  by  the  moderator. 
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Figure  46.  GMU-GcT  Obstacle  Report 


Figure  47.  Obstacle  photos  with  geotags  and  orientation  vectors 

In  order  to  explore  the  possibilities  represented  by  positional  geotags  and 
orientation  geotags  in  the  multiposition  validation  workflow,  the  authors 
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used  three  different  camera  systems  to  repeatedly  photograph  the  GMU- 
GcT  obstacle  shown  in  Figure  46.  The  three  different  camera  systems 
include  an  iPhone  6  plus,  a  Nikon  Coolpix  S9500  (an  inexpensive 
consumer-level  point  and  shoot  camera),  and  a  Canon  EOS  6D  (a 
professional-level  Digital  SLR  camera  with  a  GP-E2  GPS  attachment, 
which  captures  location  and  orientation).  All  three  devices  are  capable  of 
capturing  position  and  orientation,  and  writing  that  data  into  image 
headers,  which  are  processed  with  python  scripts  and  imported  into  a 
geographic  information  system.  The  location  and  orientation  information 
for  all  three  camera  systems  are  shown  along  with  direction  vectors  in 
Figure  48,  Figure  49,  Figure  50. 


Figure  48.  iPhone  6+  image  geotag  positions  and  orientation  vectors 


61 


Figure  49.  Nikon  CoolPix  S9500  image  geotag  positions  and  orientation  vectors 
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Figure  50.  Canon  E0S-6D  +  GP-E2  geotag  positions  and  orientation  vectors 


As  is  evident  from  the  Figures,  results  are  mixed.  The  Canon  EOS6D  +  GP- 
E2  and  Nikon  Coolpix  S9500  produce  good  results  and  show  how  geotag 
positions  and  orientation  vectors  could  be  used  to  assist  a  moderator 
confirm  the  position  of  a  report.  In  cases  where  the  contributed  report  is 
mis-positioned  due  to  contributor  error,  as  has  been  seen  in  the  GMU-GcT 
and  discussed  in  Pease  (2014)  and  Rice  et  al.  (2014),  this  additional 
element  of  validation  would  be  useful  in  performing  a  correction  to  report 
position.  The  image  geotag  positions  from  the  iPhone  6+  are  in 
approximately  the  right  places,  consistent  with  what  we  have  discussed 
about  mobile  device  GPS  constraints  in  chapter  2,  but  the  orientation 
vectors  are  incorrect,  and  in  fact,  show  a  systematic  error  in  the  wrong 
direction.  This  error  appears  to  be  a  device  and  operating  system-specific 
error  in  the  processing  of  electronic  compass  information  and  embedded 
EXIF  tags.?2  If  future  technology  has  the  same  trend  toward  improvement 


72  https://discussions.apple.com/thread/6649249?sta rt=0 
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as  it  has  in  the  past,  we  can  expect  to  see  the  same  quality  from  the  Canon 
EOS  6D  +  GP-E2  being  integrated  into  the  iPhone  and  other  less  expensive 
consumer  level  devices.  The  Nikon  Coolpix  S95000,  a  moderately  priced 
consumer  level  device  manufactured  in  2013,  has  excellent  quality  and 
good  results,  with  respect  to  image  geotag  positions  and  orientation. 

Future  incarnations  of  GMU-GcT  may  be  able  to  use  image  geotags  and 
orientation  vectors,  as  shown  here,  to  provide  a  quick  automatic  validation 
to  position. 

Geoparsed  Location  Description  Text  in  the  GMU-GcT 

Rice  et  al.  (2012a)  and  Rice  (2015)  contain  details  about  a  methods  being 
integrated  into  the  GMU-GcT,  which  involves  the  automatic  generation  of 
a  geospatial  footprint  from  the  placenames,  directions,  distances,  and 
location  descriptions  in  an  obstacle  report.  Project  collaborator  Ahmad 
Aburizaiza  has  continued  work  with  gazetteer-based  geoparsing  presented 
in  Rice  et  al.  (2011)  and  Rice  et  al.  (2012a),  and  has  substantially  increased 
its  functionality.  Using  open  source  web-mapping  tools  MapBox,  TURF.js, 
JQuery,  and  Bootstrap,  Aburizaiza  has  demonstrated  the  ability  to  quickly 
derive  a  footprint  from  geoparsed  text,  and  represent  it  on  a  map  as  a 
convex  or  concave  hull,  as  shown  in  Figure  51,  where  references  to  two 
GMU  campus  buildings  are  matched  using  a  gazetteer,  and  whose 
geometry  is  then  used  to  automatically  construct  footprints. 
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Figure  51.  Convex  (upper  left)  and  concave  (upper  right)  hull  geometries  associated  with  two 
campus  buildings.  Hull  geometry  is  refined  (bottom)  to  include  only  areas  between  the  named 

features. 


Aburizaiza  has  developed  automatic  geoparsing  code  to  construct 
footprints  for  many  generic  instances  of  obstacle  description  text, 
including  named  street  intersections  (Figure  52),  walkways  between 
named  buildings  (Figure  53),  areas  between  named  streets  and  nearby 
buildings  (Figure  54),  directional  indicators  associated  with  buildings 
(Figure  55),  and  street  segments  in  between  two  other  street  segments 
(Figure  56). 
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Figure  52.  Footprint  for  the  intersection  of  two  streets 


Figure  53.  Walkways  between  named  buildings 


Figure  54.  Footprint  for  the  area  between  a  named  street  and  a  named,  nearby  building 
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Figure  55.  Footprint  for  the  area  associated  with  a  directional  word  and  a  named  building 
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Figure  56.  Footprint  for  the  area  associated  with  a  street  segment  between  two  other  streets 

The  processing  for  the  examples  shown  can  be  done  automatically  for  any 
location  descriptions  with  named  features  from  our  gazetteer  meeting  any 
of  the  general  patterns  shown.  Augmenting  the  capability  of  the  GMU-GcT 
to  include  more  elaborate  and  complex  geospatial  relationships  and 
importantly,  to  resolve  ambiguities  when  they  arise,  is  a  goal  of  future 
work.  With  the  capabilities  presented  here  to  process  image  geotags  and 
extract  location  and  orientation,  and  to  automatically  create  footprints 
from  geoparsed  location  descriptions,  future  moderators  in  the  GMU-GcT 
will  have  many  sources  of  positioning  information  to  identify,  confirm, 
and  validate  reports.  The  final  goal  of  this  work  is  a  dashboard,  under 
development,  containing  information  such  as  Figure  57,  which  combines 
the  various  existing  forms  of  position  information  together  into  a  single 
window  that  can  be  used  by  moderators  in  their  validation  and  quality 
assessment  workflow. 
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Figure  57.  GMU-GeT  moderator  multiposition  validation  dashboard  (under  development) 
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The  next  chapter  looks  at  obstacle  engagement  with  mobile  devices,  which 
represents  an  important  augmentation  of  the  GMU-GcT  and  reflects  some 
of  the  important  dynamic  elements  of  geocrowdsourcing  discussed  in 
Chapter  2. 


70 


4  Dynamic  Obstacle  Engagement  with 
Mobile  Devices 


One  of  the  most  important  emerging  aspects  of  geocrowdsourcing  is  the 
focus  on  mobile  devices  as  both  the  platform  for  participation  by  end- 
users,  and  as  a  platform  for  authoritative  entities  to  manage,  quality  check, 
and  facilitating  geocrowdsourcing  activity.  Chapter  2  looked  at  the 
dynamic  elements  of  geocrowdsourcing,  where  with  Dr.  Dieter  Pfoser 
described  the  commoditization  of  GPS  technology  and  its  integration  in 
mobile  phones  as  a  major  element  in  dynamic  geocrowdsourcing.  This 
dynamic,  device-based  focus  has  resulted  in  vast  amounts  of  tracking  data 
that  can  be  used  in  applications  such  as  traffic  prediction  and  road 
condition  reporting.  Chapter  3  looked  at  the  ability  of  moderators  to  field 
check  reports  and  establish  position,  through  a  hybrid  mobile  and  desktop 
paradigm.  This  chapter  focuses  on  how  mobile  devices  can  be  used  for 
geocrowdsourcing  engagement  and  GMU-GcT  functionality. 

Authors  such  as  Muehlenhaus  (2013)73  see  mobile  devices  as  a  new  center 
of  information  exchange.  He  articulates  the  important  cartographic  design 
principles  for  mapping  on  mobile  devices,  including  dynamic  display 
elements  and  interfaces  that  are  “chubby-finger  friendly”  (ibid,  23). 
Dillemuth  (2005)74,  seeing  the  future  importance  of  widespread  mobile 
map  use,  designed  studies  of  pedestrian  route-following  with  mobile 
devices,  offering  cartographic  advice  and  cognitive-design  guidelines  for 
small  devices  that  would  be  used  in  dynamic,  field-based  settings.  Hupfer 
and  colleagues  (2012)75  developed  software  for  mobile  devices  to  facilitate 
simple,  rapid  construction  and  deployment  of  geocrowdsourcing 
applications.  Thatcher  (2013)76  suggests  that  the  focus  on  mobile  devices 


73  Ian  Muehlenhaus,  Web  Cartography:  Map  Design  for  Interactive  and  Mobile  Devices  (CRC  Press, 
2013). 

74  Julie  Dillemuth,  “Map  Design  Evaluation  for  Mobile  Display,"  Cartography  and  Geographic  Information 
Science  32,  no.  4  (2005):  285-301. 

75  Susanne  Hupfer  et  al.,  “MoCoMapps:  Mobile  Collaborative  Map-Based  Applications,"  in  Proceedings 
of  the  ACM  2012  Conference  on  Computer  Supported  Cooperative  Work  Companion  (ACM,  2012),  43- 
44. 

76  Jim  Thatcher,  “From  Volunteered  Geographic  Information  to  Volunteered  Geographic  Services,”  in 
Crowdsourcing  Geographic  Knowledge,  ed.  Daniel  Sui,  Sarah  Elwood,  and  Michael  F.  Goodchild 
(Dordrecht:  Springer  Netherlands,  2013),  161-73,  http://www.springerlink.com/index/10.1007/978- 
94-007-4587-2_10. 
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is  a  way  to  extend  the  static  world  of  volunteered  geographic  information 
into  a  realm  of  volunteered  geographic  services,  where  mobile  devices 
place  the  user  in  the  field  and  links  him  or  her  in  time  and  space  with 
other  users.  The  mobile  device  becomes  the  link  between  embedded  end- 
users,  who  share  volunteered  geographic  information  and  volunteered 
geographic  services. 

Haklay,  in  his  highly  cited  2010  paper  asked,  “How  good  is  volunteered 
geographic  information”?  Given  the  current  2015  context  of  what  we 
know  about  VGI  and  CGD  data  quality  (explored  in  detail  in  Girres  and 
Touya  201077,  Ruitton-Allineau  20ii78,  Rice  et  al.  20i3b79,  Rice  et  al. 
201580,  and  Rice  201581),  we  extend  the  question  to  ask,  “How  capable  are 
mobile  devices  in  facilitating  dynamic  geocrowdsourcing?”  This  focus  on 
the  mobile  platform  is  an  important  theme  in  dynamic  geocrowdsourcing 
and  explores  the  interaction  between  a  mobile  device  user  and  obstacle 
from  the  GMU-GcT. 

Dynamic  Obstacle  Engagement  in  the  GMU  Geocrowdsourcing 
Testbed 

Having  reviewed  the  uncertainty  in  obstacle  positioning  (Chapter  3)  and 
the  positioning  capabilities  of  GPS  devices  (this  chapter)  and  the 
associated  estimates  for  interaction  distances  with  obstacles  (Table  5)  the 
final  consideration  in  dynamic  obstacle  engagement  is  the  influence  that 
movement  of  the  end-user  or  moderator  will  have  in  obstacle  engagement. 
The  end-users  of  the  GMU-GcT  will  be  various,  but  likely  students,  faculty, 
and  staff  that  are  mobility  or  visually-impaired,  or  alternatively,  their  care 
givers  who  are  accompanying  them  through  an  unfamiliar  environment. 
These  end-users  need  to  be  alerted  to  the  presence  of  an  obstacle  ahead  of 
their  current  path,  so  that  they  can  avoid  it.  They  may  choose,  depending 
on  the  nature  of  the  obstacle,  to  return  and  reroute  themselves  along  a 
different  pathway.  Mobile  routing  tools,  which  will  be  discussed  at  the  next 


77  Girres  and  Touya,  “Quality  Assessment  of  the  French  OpenStreetMap  Dataset.” 

78  Anne-Marthe  Ruitton-Allinieu,  “Crowdsourcing  of  Geoinformation:  Data  Quality  and  Possible 
Applications”  (Master  of  Science,  Aalto  University,  2011), 
http://maa.aalto.fi/fi/geoinformatiikan_tutkimusryhma- 
gma/geoinformatiikkaja_kartografia/2011_ruitton-allinieu_a.pdf. 

79  Rice  et  al.,  “Crowdsourcing  to  Support  Navigation  for  the  Disabled:  A  Report  on  the  Motivations, 
Design,  Creation  and  Assessment  of  a  Testbed  Environment  for  Accessibility.” 

80  Rice  et  al.,  “Position  Validation  in  Crowdsourced  Accessibility  Mapping.” 

81  Rice,  “Validating  VGI  Data  Quality  in  Local  Crowdsourced  Accessibility  Mapping  Applications:  A  George 
Mason  University  Case  Study.” 
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chapter,  allow  for  obstacle-avoidance  routing.  Because  the  end-users  of 
our  system  are  moving  through  space,  the  distance  required  to  alert  them 
of  an  obstacle  ahead  of  them  increases  as  their  own  speed  increases.  This 
dynamic  suggests  that  we  need  to  increase  the  obstacle  engagement 
distances  required  for  dynamic  engagement  (Table  5).  While  conducting 
field  tests  in  July,  and  August,  we  experimented  with  various  interaction 
distances  based  on  known  GPS  characteristics  and  based  on  the 
information  in  Table  5  During  testing  in  early  August,  we  established  100 
feet  (30.48  meters)  as  an  acceptable  reflection  of  the  uncertainties 
reflected  in  Table  5,  as  well  as  the  impact  that  a  moving  end-user  would 
have  on  the  necessary  obstacle  alert  distance. 

A  preliminary  study  by  Eric  Ong  (Figure  58)  for  the  pedestrian  network  on 
the  George  Mason  University  campus  (our  area  of  most  significant 
activity)  indicated  an  average  segment  length  of  87.59  meters,  with  a 
majority  of  segments  under  20  meters.  An  interaction  distance  of  100  feet 
(30.48  meters)  would  in  some  cases,  result  in  an  end-user  needing  to 
backtrack,  but  during  testing  the  distance  seemed  to  be  a  good 
compromise  for  all  known  factors  and  practical  usability  issues. 
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Figure  58  Segment  lengths  in  meters  for  the  George  Mason  University  campus 

The  only  unknown  factor,  after  accepting  general  interaction  distances 
based  on  multiple  sources  of  uncertainty,  was  the  latency  and 
responsiveness  of  the  computer  data  networks,  server,  and  mobile 
software.  These  issues  will  be  discussed  in  the  context  of  our  field  work. 
The  next  section  of  this  chapter  discusses  two  applications  for  mobile 
obstacle  engagement  developed  as  a  part  of  this  work,  and  a  set  of  field 
tests  to  measure,  visualize,  and  analyze  patterns  in  mobile  obstacle 
engagement. 
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Mobile  applications  for  obstacle  engagement 

The  GMU-GcT  end  users  can  access  our  GMU-GcT  obstacle  information 
online  through  our  web  portal82,  but  also  through  a  new  native  mobile 
application  developed  by  project  researcher  Samuel  Ober  (Figure  59).  This 
native  mobile  application  was  developed  from  earlier  incarnations  coded 
with  Sencha  Touch,  but  has  been  updated  using  Swift,  an  open-source 
coding  tool  for  iOS.  Swift  has  a  concise  syntax  that  generates  faster  code 
for  mobile  devices  with  low-power  processers.  The  current  version  is  of  the 
GMU-GcT  Mobile  App  has  been  tested  by  moderators  and  will  be  released 
through  Apple’s  App  Store  pending  final  approval.  This  native  mobile 
application  can  take  advantage  of  the  wide  range  of  functions  in  iOS,  and 
has  a  benefit  of  running  continuously,  whether  or  not  it  is  open  and 
maximized  on  screen.  An  alternative  mobile  web  application,  developed  by 
project  researcher  Ahmad  Aburizaiza,  uses  Mapbox,  Turf,  and  MongoDB, 
along  with  existing  functionality  from  the  desktop  version  of  the  GMU- 
GcT  (Figure  60).  This  mobile  web  application  has  an  advantage  of  being 
able  to  utilize  the  large  library  of  spatial  functions  from  Turf.  The  use  of  a 
NoSQL  database  (MongoDB)  has  significant  performance  benefits  relative 
to  the  relational  database  used  in  the  GMU-GcT  and  during  preliminary 
testing  had  quick,  rapid  performance. 


82  http://geo.gmu.edu/vgi/  [accessed  November  19,  2015] 
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Figure  59.  Native  mobile  application  for  obstacle  engagement,  developed  with  Swift  for  iOS 
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Figure  60.  Web  Application  for  mobile  obstacle  interaction,  developed  with  Turf/Mapbox  and 

MongoDB 


Field  studies  of  obstacle  engagement 

Between  July  1  and  present,  the  two  different  GMU-GcT  mobile 
applications,  referred  to  hereafter  as  the  MobileApp  (for  Samuel  Ober’s 
Swift-based  iOS  native  app,  Figure  59),  and  WebApp  (for  Ahmad 
Aburizaiza’s  Turf/Mapbox  web  application,  Figure  60)  were  tested 
extensively,  to  gauge  the  ability  of  a  mobile  user  to  receive  alerts  from  each 
of  the  applications  to  notify  him  or  her  about  the  presence  of  an  obstacle. 

A  study  location  close  to  the  GMU  campus  provided  access  to  a  dynamic, 
construction-filled  environment  with  several  obstacles  that  had  been 
reported  to  the  GMU-GcT  repeatedly.  Two  obstacles  were  chosen  and 
selected  for  a  series  of  tests  in  which  the  end-user  would  move  toward  the 
obstacle  from  different  directions.  Research  assistants  would  mark  the 
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exact  location  where  an  obstacle  alert  was  delivered  to  the  end-user’s 
mobile  device,  and  would  then  measure  the  distance  between  this  alert 
position  and  the  obstacle.  In  order  to  address  variables  such  as  canopy 
cover  and  building  heights,  one  obstacle  was  chosen  in  close  proximity  to  a 
3.5-story  row  of  townhouses  with  an  approximate  height  of  48  feet  and 
covered  with  a  tree  canopy.  Another  obstacle  was  chosen  on  a  walkway 
adjacent  to  a  single-story  medical  facility,  with  trees  nearby  in  an  area 
characterized  as  Partial  Canopy. 

Based  on  user  needs  assessments  conducted  for  this  project  and  discussed 
in  Rice  et  al.  (2014),  we  anticipate  that  end-users  and  moderators  will 
need  to  be  alerted  to  obstacles  while  moving  through  daily  activities  at  the 
rates  of  speed  similar  to  walking.  Moderators,  who  may  be  riding  a  bicycle 
or  moving  at  a  faster  speed,  will  have  engagement  distances  with  an  even 
greater  required  alert  distance.  To  explore  the  delivery  of  obstacle  alerts  to 
a  moving  end-user,  a  mobile  data  collection  platform  was  constructed 
from  a  bicycle  with  camera  and  mobile  device  mounts.  This  allows  an  end- 
user  to  interact  with  a  mobile  application  while  riding,  and  film  the 
interaction  with  a  forward  looking  GoPro  camera.  A  compact  screen¬ 
capturing  program  running  on  the  rear-facing  mobile  device  records 
mobile  alerts  and  user  interaction  with  the  mobile  applications.  Figure  61 
shows  this  platform,  which  has  an  additional  USB  solar  charging  capacity 
for  mobile  devices  and  a  small  Bluetooth  speaker  for  auditory  cues  (not 
pictured). 
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Figure  61.  GMU-GcT  Mobile  Data  Collection  Platform,  with  LTE-enabled  mobile  device  and 
forward  facing  video  capture  capability 


Obstacle  engagement  dynamics  with  the  GMU-GcT  mobile  WebApp 

The  mobile  web  application,  or  WebApp  (Figure  60),  was  coded  using 
Mapbox/Turf.  It  uses  data  and  functionality  from  the  GMU-GcT  and  uses 
the  PostgreSQL  database,  but  copied  to  a  MongoDB  NoSQL  database  to 
increase  dynamic  performance.  The  WebApp  is  responsive  and  performed 
well  in  preliminary  testing.  Because  communication  between  the  WebApp 
and  the  server  is  more  verbose  and  frequent,  we  anticipated  that  there 
would  be  performance  during  dynamic  field  testing.  For  all  dynamic  field 
testing,  we  used  four  different  mobile  devices:  an  iPhone  5,  and  iPhone  6, 
an  iPhone  6+,  and  an  iPad2.  All  devices  were  capable  of  Long-Term 
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Evolution  (LTE)  data  speeds  and  all  devices  used  the  same  mobile  service 
provider  (AT&T).  The  two  obstacles  used  in  field  testing  are  shown  along 
with  data  summaries  in  Figure  62  and  Figure  63.  The  field  testing  protocol 
involved  the  following  steps: 

1.  The  obstacle  is  verified  in  the  GMU-GcT.  Any  discrepancies  or 
errors  are  corrected  before  field  testing  begins. 

2.  The  obstacle’s  ground  center  point  is  marked  with  tape. 

3.  The  WebApp  (or  MobileApp)  is  loaded,  and  screen  capture  is 
initiated. 

4.  The  forward  facing  camera  is  enabled  and  started  for  video  capture. 

5.  The  obstacle  is  approached  at  walking  speed  (2-3  mph),  along  one 
of  eight  pre-defined  and  marked  directional  trajectories  (see 
Figures  58  and  59). 

6.  The  exact  ground  point  where  an  obstacle  alert  is  received  on  the 
mobile  device  is  marked  with  tape  along  the  trajectory. 

7.  The  dynamic  alert  distance  is  measured  and  recorded. 

8.  The  process  (steps  1-6)  is  repeated  for  each  of  the  four  mobile 
devices,  along  each  of  the  eight  pre-defined  trajectories. 

9.  The  process  (steps  1-7)  is  repeated,  but  at  a  higher  rate  of  speed  (7- 
8  mph)  using  the  GMU-GcT  mobile  data  collection  platform  (Figure 
57). 

10.  The  entire  process  (steps  1-8)  is  repeated  using  the  MobileApp  in 
place  of  the  WebApp. 

With  this  protocol  in  place,  field  testing  was  begun.  Figure  62  shows 
engagement  with  a  sidewalk  obstruction  consisting  of  an  orange  barrel, 
which  in  later  phases  was  switched  by  the  construction  workers  to  an 
orange  cone.  This  barrel  and  cone  were  being  used  as  a  marker  for 
modifications  to  the  curb  and  roadway  for  the  nearby  Eleven  Oaks 
development.  Underneath  the  obstacle  picture  and  summary  in  Figure  62 
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and  Figure  63,  obstacle  interaction  distances  and  diagrams  for  a  bicycling 
end-user  (top),  and  for  a  walking  end-user  (bottom)  are  shown.  The  eight 
trajectories  for  each  travel  mode  are  numbered  and  marked  in  black,  with 
color-coded  markers  showing  the  obstacle  alert  location  and  lines 
connected  the  color-coded  points  to  allow  for  interpretation  of  shape  and 
general  device-based  interaction  dynamics.  We  also  record  the  average 
interaction  distance  for  each  device  and  for  each  mode  of  travel,  along  with 
a  standard  deviation.  The  interaction  distances  consist  of  two 
measurements.  First,  we  measure  the  distance  between  the  end-user  and 
the  obstacle,  which  is  summarized  as  ‘average  distance  to  obstacle’. 

Second,  we  record  the  distance  between  the  100-foot  buffer  (shown  as  a 
grey  arc  on  the  diagrams)  and  the  user,  which  is  summarized  as  ‘average 
distance  from  buffer’.  These  distances  sum  to  100  feet,  which  is  the  total 
distance  between  the  obstacle  and  the  100  foot  buffer. 

For  both  obstacles  for  all  devices,  (Figure  62  and  Figure  63)  the  obstacle 
alert  distances  for  a  walking  end-user  are  greater  than  for  a  biking  end- 
user,  which  was  an  expected  result  related  to  the  nature  of  dynamic 
interaction  and  the  speed  with  which  the  WebApp  would  be  able  to  detect 
and  notify  the  end  user.  The  alternative  view  of  the  interaction  dynamics 
are  that  the  walking  end-user  generally  receives  an  obstacle  alert  closer  to 
the  intended  100  foot  buffer  than  the  bicycling  end  user.  The  100  foot 
buffer  and  a  summary  of  both  interaction  distances  are  contained  in  all 
obstacle  interaction  figures. 

For  Figure  62  and  Figure  63,  the  obstacle  alert  distances  increase  as  the 
trajectory  moves  away  from  the  buildings  on  the  north  side  (Figure  62) 
and  south  sides  of  the  road  (Figure  63).  The  fan  shape  in  Figure  63  is 
prominent,  with  trajectories  4  and  5  having  larger  interaction  distances 
than  other  trajectories,  which  is  likely  related  to  the  decrease  in  multipath 
error  and  reflection  for  positions  farther  from  the  buildings.  There  is  no 
apparent  ordinal  effect  related  to  devices,  which  all  delivered  alerts  in 
close  proximity  to  one  another,  with  a  few  exceptions  (iPhone  6,  biking 
trajectories  2  and  5  in  Figure  62,  and  the  iPad  2  trajectory  5  in  Figure  63). 
The  iPhone  4,  used  in  earlier  mobile  GPS  studies,  was  not  used  during  our 
mobile  obstacle  interaction  study.  The  iPhone  6+  was  not  tested  earlier 
with  Frechet  distances,  but  was  used,  along  with  the  iPad  2,  in  our  mobile 
obstacle  interaction  study.  Our  goal  was  to  test  and  experiment  with  a 
variety  of  mobile  devices,  and  to  explore  their  limitations  under  a  variety 
of  use  scenarios  and  environmental  conditions.  The  testing  protocols 


80 


presented  in  this  chapter  and  in  Chapter  2  will  be  continued  with  a  variety 
of  devices  as  they  become  available,  and  will  help  us  augment  our 
knowledge  of  mobile  GPS  capabilities  and  how  these  capabilities  influence 
mobile  geocrowdsourcing. 

For  the  obstacle  engagements  studies  presented  here,  we  used  an 
interaction  distance  of  100  feet  (shown  in  following  figures  with  a 
darkened  gray  circle).  None  of  the  alert  distances  for  the  WebApp  are  close 
to  that  distance,  with  the  largest  alert  distance  being  79.3  feet  (iPhone  6+, 
Figure  63).  The  total  range  of  alert  distances  for  the  WebApp  was  48.1  to 
79.3  feet.  Figure  64  and  Figure  65  show  the  same  obstacle  alert  distances, 
but  broken  down  by  device.  Side  by  side  comparisons  of  the  average 
distance  ellipses  and  obstacle  alert  distances  clearly  show  a  decrease 
between  walking  and  biking. 
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Obstacle  Details 

Type:  Sidewalk  Obstruction 

Location:  School  St.  sidewalk  near  one  of  the 
entrances  to  Inova  Care  Center,  west  of 
George  Mason  Blvd. 

Description:  Orange  barrel  partially  blocking 
sidewalk 


A  iPhone  5 

O  iPhone  6 

□  iPhone  6+ 

O  iPad  2 

Average  Distance  to  Obstacle  (ft): 

50.2 

58.3 

55.1 

48.1 

Average  Distance  from  Buffer  (ft): 

49.8 

41.7 

44.9 

51.9 

Standard  Deviation  (ft): 

11.9 

11.5 

14.6 

9.4 

A  iPhone  5 

O  iPhone  6 

□  iPhone  6+ 

O  iPad  2 

Average  Distance  to  Obstacle  (ft): 

69.1 

67.4 

72.7 

67.8 

Average  Distance  from  Buffer  (ft): 

30.9 

32.6 

27.3 

32.2 

Standard  Deviation  (ft): 

10.8 

14.6 

12.6 

10.3 

Figure  62.  Dynamic  obstacle  engagement  with  WebApp,  obstacle  367  summary 


82 


Obstacle  Details 

Type:  Sidewalk  Obstruction 
Construction  Detour 

Location:  On  School  St.  on  the  sidewalk 
across  from  the  Commonwealth  Care  Center 

Description:  Sidewalk  excavation. 

Closed  Sidewalk. 


A  iPhone  5 

O  iPhone  6 

□  iPhone  6+ 

O  iPad  2 

Average  Distance  to  Obstacle  (ft): 

63.1 

61.8 

67.8 

69.4 

Average  Distance  from  Buffer  (ft): 

36.9 

38.2 

32.2 

30.6 

Standard  Deviation  (ft): 

5.5 

7.5 

7.7 

10.2 

A  iPhone  5 

O  iPhone  6 

□  iPhone  6+ 

O  iPad  2 

Average  Distance  to  Obstacle  (ft): 

79.1 

76.4 

79.3 

76.6 

Average  Distance  from  Buffer  (ft): 

20.9 

23.6 

20.7 

23.4 

Standard  Deviation  (ft): 

8.4 

8.4 

8.9 

7.7 

Figure  63.  Dynamic  obstacle  engagement  with  the  WebApp,  obstacle  11  summary 
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average  distance  to  obstacle  =  50.2  ft 
I  average  distance  from  buffer  =  49.8  ft 


average  distance  to  obstacle  =  69.1  ft 
average  distance  from  buffer  =  30.9  ft 
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A  iPhone  5  •  iPhone  6 


iPhone  6+4iPad  2 


average  distance  to  obstacle  =  67.4  ft 
average  distance  from  buffer  =  32.6  ft 


average  distance  to  obstacle  =  58.3  ft 
average  distance  from  buffer  =  41 .7  ft 


average  distance  to  obstacle  =  72.7  ft  H 
average  distance  from  buffer  =  27.3  4 


average  distance  to  obstacle  =  55.1  ft  ~ 
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- 


average  distance  to  obstacle  =  48.1  ft 
average  distance  from  buffer  =  51 .9  ft 


average  distance  to  obstacle  =  67.8  ft 
average  distance  from  buffer  =  32.2  ft 


■  r 


Figure  64.  Dynamic  obstacle  engagement  with  WebApp,  obstacle  367  by  device 
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WalKinq~  MA  iPhone  5  •iPhone  6  ■  iPhone  6+  ♦iPad  2 


A  iPhone  5 


iPhone  6 


iPhone  6+  ♦iPad  2  p*  J 


average  distance  to  obstacle  =  63.1  ft 
average  distance  from  buffer=  36.9  ft 


average  distance  to  obstacle  =  79.1  ft 
average  distance  from  buffer=  20.9  ft 


average  distance  to  obstacle 
average  distance  from  buffer 


61.8  ft 


average  distance  to  obstacle  =  76.4  ft 
average  distance  from  buffer=  23.6  ft 


38.2  ft 


. 


average  distance  to  obstacle  =  79.3  ft 
average  distance  from  buffer=  20.7  ft 


average  distance  to  obstacle 
average  distance  from  buffer 


67  3  tt 


32.2  ft 


average  distance  to  obstacle  =  76.6  ft 
average  distance  from  buffer=  23.4  ft 


average  distance  to  obstacle 
average  distance  from  buffer 


30.6  ft 


Figure  65.  Dynamic  obstacle  engagement  with  WebApp,  obstacle  11  by  device 

Obstacle  engagement  dynamics  with  the  GMU-GcT  MobileApp 

The  same  field  testing  protocol  discussed  above  was  employed  to  test  the 
functionality  and  obstacle  alert  distance  for  the  native  MobileApp,  which 
we  anticipated  to  be  more  responsive  due  to  the  less  verbose 
communication  required,  and  an  assumption  that  for  the  same  iOS 
devices,  latency  would  be  lower  for  similar  tasks.  Jobe  (2013)  compared 
native  mobile  and  mobile  web  applications  for  map-based  athletic  tracking 
applications  in  Kenya,  finding  that  “mobile  web  applications  that  require 
hardware  interaction  such  as  using  the  GPS,  GPU,  or  camera  are  not  yet 
viable  alternatives  for  native  applications.”  He  suggests  that  simpler 
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mobile  web  applications  “that  only  require  a  native  interface  and  content 
consumption  are  suitable  substitutes  for  native  applications.”  Clearly,  our 
WebApp  does  need  access  to  the  device  GPS  and  is  much  more  than  a 
simple  reduction  of  a  desktop  application  to  a  native  interface,  and  based 
on  this,  we  did  expect  to  see  differences.  Our  purpose  in  this  study, 
however,  is  not  to  compare  the  GMU-GcT  WebApp  with  the  GMU-GcT 
MobileApp,  but  rather  to  look  at  what  is  possible  with  the  two,  recognizing 
the  strengths  of  each  and  assessing  what  benefits  each  might  provide  to  for 
an  end-user. 

With  this  in  mind,  Figure  66  and  Figure  67  (similar  to  Figure  62  and 
Figure  63)  contain  summaries  of  the  obstacle  alert  distances  for  user- 
engagement  with  the  same  4  mobile  devices,  the  same  2  obstacles,  and  the 
same  8  pre-defined  trajectories.  Figure  68  and  Figure  69  contain  the  same 
side-by-side  device  and  travel  mode  comparisons.  An  inspection  of  the 
figures  and  the  100  foot  gray  interaction  distance  quickly  leads  to  the 
conclusion  that  the  native  app  is  much  better  at  delivering  obstacle  alerts 
closer  to  the  intended  100  foot  interaction  distance. 

For  reasons  that  we  are  not  able  to  yet  explain,  there  are  some  directional 
asymmetries  in  the  obstacle  alert  distances,  where  for  Obstacle  367  (Figure 
66)  all  four  devices  delivered  alerts  quicker  on  the  west  side  (eastbound 
trajectories)  than  on  the  eastside  (westbound  trajectories).  Similarly,  in 
Figure  67  (biking  mode)  the  MobileApp  delivered  obstacle  alerts  more 
quickly  on  the  east  side  than  on  the  west  side.  In  summary,  all  obstacle 
alert  distances  were  relatively  close  to  the  100  foot  interaction  distance, 
with  the  walking  trials  being  nearly  identical  for  many  devices.  This  is 
clearly  seen  in  Figure  68  and  Figure  69,  where  the  colored  average 
obstacle  alert  distance  circles  entirely  (or  nearly  entirely)  overlap  the  gray 
100  foot  interaction  distance  circle. 

For  walking  end-users  who  approach  obstacles  slowly,  this  suggests  that 
there  will  be  no  significant  difference  depending  on  the  device  they  use, 
and  that  alerts  should  be  delivered  relatively  close  to  the  100  foot 
interaction  distance.  For  bicycling  end-users,  obstacle  alerts  are  delivered 
further  away  from  the  intended  100  foot  distance,  and  again,  there  is  no 
statistically  significant  difference  for  which  device  they  may  be  using  to 
receive  alerts,  at  least  among  the  four  that  we  tested. 
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Obstacle  Details 

Type:  Sidewalk  Obstruction 

Location:  School  St.  sidewalk  near  one  of  the 
entrances  to  Inova  Care  Center,  west  of 
George  Mason  Blvd. 

Description:  Orange  barrel  partially  blocking 
sidewalk 


A  iPhone  5 

O  iPhone  6 

□  iPhone  6+ 

O  iPad  2 

Average  Distance  to  Obstacle  (ft): 

85.7 

85.1 

84.0 

80.8 

Average  Distance  from  Buffer  (ft): 

14.3 

14.9 

16.0 

19.2 

Standard  Deviation  (ft): 

6.6 

6.5 

6.5 

6.2 

A  iPhone  5 

Q  iPhone  6 

□  iPhone  6+ 

O  iPad  2 

Average  Distance  to  Obstacle  (ft): 

93.5 

98.6 

99.8 

99.6 

Average  Distance  from  Buffer  (ft): 

6.5 

1.4 

0.2 

0.4 

Standard  Deviation  (ft): 

12.0 

13.0 

13.0 

11.0 

Figure  66.  Dynamic  obstacle  engagement  with  MobileApp,  obstacle  367  summary 
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A  iPhone  5  O  iPhone  6  □  iPhone  6+  O  iPad  2 


Average  Distance  to  Obstacle  (ft): 

93.6 

92.6 

91.6 

95.3 

Average  Distance  from  Buffer  (ft): 

6.4 

7.4 

8.4 

4.7 

Standard  Deviation  (ft): 

7.3 

7.2 

7.1 

7.4 

A  iPhone  5 

O  iPhone  6 

□  iPhone  6+ 

OiPad : 

Average  Distance  to  Obstacle  (ft): 

85.4 

88.3 

86.6 

78.5 

Average  Distance  from  Buffer  (ft): 

14.6 

11.7 

13.4 

21.5 

Standard  Deviation  (ft): 

7.2 

7.4 

5.8 

6.1 

Obstacle  Details 


Type:  Sidewalk  Obstruction 
Construction  Detour 


Location:  On  School  St.  on  the  sidewalk 
across  from  the  Commonwealth  Care  Center 


Description:  Sidewalk  excavation. 
Closed  Sidewalk. 


Figure  67.  Dynamic  obstacle  engagement  with  MobileApp,  obstacle  11  summary 
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distance  from  buffer  =  20.0  ft 


average  distance  to  obstacle  =  80.0  ft 


m 


A  iPhone  5  •  iPhone  6  ■  iPhone  6+  4iPad 


walkino^^A  lPhone  5  •'Phone  6  ■  iPhone  6+»iPad  2 


bikin 


|  average  distance  to  obstacle  =  93.5  ft  average  distance  from  buffer  =  6.5  ft  average  distance  to  obstacle  =  85.7  ft  average  distance 


from  buffer  =  14.3  ft 


average  distance  to  obstacle  =  98.6  ft  average  distance  from  buffer  =  1 .4  ft 


average  distance  to  obstacle  =  85.1  ft  average  distance  from  buffer  =  14.9  ft 


average  distance  to  obstacle  =  99.8  ft  average  distance  from  buffer 


average  distance  to  obstacle  =  84.0  ft  average  distance  from  buffer  =  16.0  ft 


2- 


average  distance  to  obstacle  =  99.6  ft  average  distance  from  buffer 


average 


Figure  68.  Dynamic  obstacle  engagement  with  MobileApp,  obstacle  367  by  device 
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| A  iPhone  5  m  iPhone  6  ■  iPhone  6+ ♦iPad  2  [ 


|a  iPhone  5  •  iPhone  6  ■  iPhone  6+AiPad  2| 


|  average  distance  to  obstacle  =  85.4  ft  average  distance  from  buffer  =  1 4.6  ft  [ 


|  average  distance  to  obstacle  =  93.6  ft  average  distance  from  buffer  =  6.4  ft 


|  average  distance  to  obstacle  =  88.3  ft  average  distance  from  buffer  =  1 1 .7  ft 


average  distance  to  obstacle  =  92.6  ft  average  distance  from  buffer  =  67.4  ft 


|  average  distance  to  obstacle  =  91 .6  ft  average  distance  from  buffer  =  8.4  ft| 


average  distance  to  obstacle  =  86.6  ft  average  distance  from  buffer  =  13.4  ft 


|  average  distance  to  obstacle  =  95.3  ft  average  distance  from  buffer  =  4.7  ft 


average  distance  to  obstacle  =  78.5  ft  average  distance  from  buffer  =  21 .5  ft 


Figure  69.  Dynamic  obstacle  engagement  with  MobileApp,  obstacle  11  by  device 


Statistical  Summaries  for  Mobile  Obstacle  Engagement 

Engagement  with  the  WebApp 

Statistical  analysis  of  the  variables  and  factors  in  the  WebApp  portion  of 
the  field  study  indicate  that  the  influence  of  device  (iPhone  5,  iPhone  6, 
iPhone  plus,  iPad  2)  was  not  significant  when  using  the  WebApp  (Figure 
60).  The  associated  analysis  of  variance  (ANOVA)  for  device  resulted  in  an 
F-statistics  of  0.43361  with  p-value  =  0.7293,  at  3  and  124  degrees  of 
freedom  (Table  10).  Although  small  variations  existed  between  devices 
during  the  study,  no  systematic,  significant  differences  existed.  The 
graphics  presented  on  the  preceding  pages  do  seem  to  show  a  difference  in 
the  obstacle  engagement  distances  for  walking  and  biking,  with  the 
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distance  between  the  end-user  and  obstacle  being  smaller  when  bicycling, 
This  is  associated  with  a  larger  distance  between  the  intended  loo-foot 
warning  buffer  (shown  in  gray  in  Figure  62,  Figure  63,  Figure  64,  Figure 
65)  and  the  point  where  an  obstacle  alert  was  delivered  by  the  WebApp. 

Table  10.  Analysis  of  Variance  for  Mobile  Device  using  WebApp 


Analysis  of  Variance  for  Mobile  Device  (One-Way) 

Summary 

Groups 

Sample 

size 

Sum 

Mean 

Variance 

iPhone  5 

32 

2,091.75 

65.36719 

194.58551 

iPhone  6 

32 

2,110.41667 

65.95052 

154.86239 

iPhone  6plus 

32 

2,199.41667 

68.73177 

195.73465 

iPad2 

32 

2,095.66667 

65.48958 

196.13026 

ANOVA 

Source  of  Variation 

SS 

df 

MS 

F 

p-level 

Fcrit 

Between  Groups 

241.08219 

3 

80.36073 

0.43361 

72.9309% 

2.6777 

Within  Groups 

22,980.69727 

124 

185.3282 

Total 

23,221.77946 

127 

After  a  preliminary  test  for  the  presence  of  homogeneity  or 
heteroskedasticity  in  the  sample  variances  (Table  11),  a  T-test  for  equality 
of  means  was  conducted,  resulting  in  a  large,  statistically  significant  t- 
value  of  7.037  with  p-value  =  0.00  and  t-critical  value  of  1.97  (Table  12). 
This  suggests  that  the  difference  between  the  bicycling  engagement 
distance  and  the  walking  engagement  distances  for  the  WebApp  is 
statistically  significant.  End-users  moving  at  a  walking  pace  (2-3  mph)  and 
using  the  WebApp  receive  obstacle  warnings  much  closer  to  the  100-foot 
interaction  buffer  than  end-users  who  are  moving  at  a  faster  rate  on  a 
bicycle  (7-8  mph). 


Table  11.  F-testfor  Homogeneity  of  Variances,  Walking  and  Biking 


F-Test  Two-Sample  for  Variances:  Walking  and  Biking 

Descriptive  Statistics 

VAR 

Walking 

Biking 

Sample  size 

64 

64 

Mean 

73.54219 

59.23047 

Variance 

119.01365 

145.65978 

Standard  Deviation 

10.90934 

12.06896 

Mean  Standard  Error 

1.36367 

1.50862 

Summary 

F 

1 .22389  F  Critical  value  (5%) 

1.51833 

p-level  1 -tailed 

0.21245  p-level  2-tailed 

0.4249 

HO  (5%)? 

accepted 
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Table  12.  T-test  for  Equality  of  Means,  Walking  and  Biking 


T-test  for  Equality  of  Means,  Walking  and  Biking 

Descriptive  Statistics 

VAR 

Sample  size 

Mean 

Standard  Deviation 

Variance 

Walking 

64 

73.54219 

10.90934 

119.01365 

Biking 

64 

59.23047 

12.06896 

145.65978 

t-test  assuming  equal  variances  (homoscedastic) 

Degrees  of  Freedom 

126 

Hypothesized  Mean  Difference 

0. 

Pooled  Variance 

132.33671 

Test  Statistics 

7.03763 

Two-tailed  distribution 

p-level 

1.11397E-10  Critical  Value  (5%) 

1.97897 

One-tailed  distribution 

p-level 

5.56983E-11  Critical  Value  (5%) 

1.65704 

Finally,  a  test  for  the  two  obstacles  shown  in  this  field  study  indicates  that 
the  interaction  distances  for  Obstacle  n  (Figure  63,  Figure  65,  Figure  67, 
Figure  69)  are  significantly  larger  (with  an  observed  average  of  71.7 
meters)  than  for  Obstacle  367  (Figure  62,  Figure  64,  Figure  66,  Figure  68), 
where  an  average  value  of  61.1  was  observed.  The  statistical  analysis  for 
this  difference  is  shown  in  Table  13  and  Table  14.  The  reasons  for  this 
difference  are  uncertain,  and  no  clear  contributing  factors  have  been 
identified. 

Future  work  on  mobile  device-based  obstacle  engagement  with  the 
WebApp  or  similar  technology  would  test  obstacles  in  a  wide  variety  of 
settings  under  a  number  of  different  conditions  and  engagement 
dynamics.  The  latency  associated  with  the  WebApp,  discussed  earlier,  may 
present  itself  as  a  temporally-significant  covariate,  where  network 
conditions  vary  significantly  throughout  the  testing  period,  impacting  the 
ability  of  the  WebApp  and  leading  to  temporally-significant  dynamics. 
Because  absolute  time  was  not  recorded  as  a  factor,  we  cannot  test  for  this 
possible  source  of  variation,  but  it  would  be  a  part  of  future  work. 
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Table  13:  F-testfor  Homogeneity  of  Variances:  Obstacle  367  and  Obstacle  11 


F-Test  Two-Sample  for  Variances  (Obstacle  367  &  0411) 

Descriptive  Statistics 

VAR 

Obstacle  0411  Obstacle  367 

Sample  size 

Mean 

Variance 

Standard  Deviation 

Mean  Standard  Error 

64  64 

71.6849  61.08464 

103.32352  208.20161 

10.16482  14.42919 

1.2706  1.80365 

Summary 

\F 

p-level  1 -tailed 

HO  (5%)? 

2.01505  F  Critical  value  (5%) 

0.00305  p-level  2-tailed 

rejected 

1.51833 

0.0061 

Table  14.  Test  for  Equality  of  Means,  Obstacle  367  and  Obstacle  11 


Students  T-test  for  Equalityj>f  Means^  Obstacle  367  and  Obstacle  0411^ 


Descriptive  Statistics 

VAR 

Sample  size 

Mean 

Standard  Deviation 

Variance 

Obstacle  367 

64 

61.08464 

14.42919 

208.20161 

Obstacle  0411 

64 

71.6849 

10.16482 

103.32352 

t-test  assuming  unequal  variances  (heteroscedastic) 


Degrees  of  Freedom  113 

Hypothesized  Mean  Difference  0. 

Pooled  Variance  155.76256 

Test  Statistics  4.80463 


Two-tailed  distribution 

p-level_ 4.80359E-6  Critical  Value  (5%) 1.98118 


Engagement  with  the  MobileApp 

Statistical  analysis  of  the  field  testing  and  obstacle  interactions  using  the 
MobileApp  (Figure  59)  yielded  results  that  were  similar  in  some  ways  and 
different  in  other  ways  from  the  results  of  the  field  testing  for  the  WebApp. 
As  noted  previously,  the  MobileApp  has  a  more  responsive  design,  and 
because  it  is  deeply  integrated  with  the  mobile  operating  system,  it  can 
take  advantage  of  interactions,  cues,  and  alert  functions  from  the 
operating  system.  These  same  features  are  not  possible  with  the  WebApp. 
Although  these  advantages  do  not  always  lead  to  more  responsive 
performance,  field  testing  with  the  native  MobileApp  quickly  led  to  a 
hypothesis  that  interaction  times  would  be  quicker. 

As  a  first  step  in  our  statistical  analysis,  an  ANOVA  for  device  type  resulted 
in  a  conclusion  of  no  significant  differences  (Table  15).  The  F-value  for  this 
test  was  0.354  with  a  p-value  of  0.7862  and  an  associated  F-critical  value 
of  2.6777,  with  3  and  124  degrees  of  freedom.  In  summary,  none  of  the 
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mobile  devices  resulted  in  interaction  distances  that  were  significantly 
different.  Small  differences  in  interaction  times  were  observed  and  are 
presented  in  Figure  62,  Figure  63,  Figure  64,  and  Figure  65,  but  these 
differences  are  not  statistically  significant.  In  this  particular  aspect,  both 
the  WebApp  and  MobileApp  had  the  same  statistical  result  of  no 
significant  mobile  device  differences. 


Table  15.  Analysis  of  Variance  for  Mobile  Device  using  MobileApp 


Analysis  of  Variance  (One-Way) 


Summary 


Groups 

Sample  size 

Sum 

Mean 

Variance 

iPhone  5 

32 

2,865.25 

89.53906 

92.63888 

iPhone  6 

32 

2,916.75 

91.14844 

104.17551 

iPhone  6plus 

32 

2,896.41667 

90.51302 

126.17881 

iPad2 

32 

2,832.83333 

88.52604 

152.67806 

ANOVA 

Source  of  Variation 

SS 

df 

MS 

F 

p-level 

Fcrit 

Between  Groups 

126.34956 

3 

42.11652 

0.35416 

78.62019% 

2.6777 

Within  Groups 

14,745.80924 

124 

118.91782 

Total 

14,872.1588 

127 

Next,  after  a  preliminary  test  for  homogeneity  of  sample  variances  (Table 
16),  a  T-test  for  equality  of  means  was  conducted,  resulting  in  large, 
statistically  significant  T-value  of  6.894,  with  p-value  0.000  and  an 
associated  T  critical  value  of  1.97  (Table  17).  This  suggests  that  the 
difference  between  the  bicycling  engagement  distance  and  the  walking 
engagement  distances  for  the  MobileApp  is  statistically  significant,  similar 
to  the  WebApp.  End-users  moving  at  a  walking  page  receive  obstacle 
notifications  much  closer  to  the  100-foot  interaction  buffer. 


Table  16.  F-testfor  Homogeneity  of  Variances,  Walking  and  Biking 


F-Test  Two-Sample  for  Variance:  Walking  and  Biking 

Descriptive  Statistics 

VAR 

Biking 

Walking 

Sample  size 

64 

64 

Mean 

84.29036 

95.57292 

Variance 

85.24912 

86.15862 

Standard  Deviation 

9.23304 

9.28217 

Mean  Standard  Error 

1.15413 

1.16027 

Summary 

F 

1 .01067  F  Critical  value  (5%) 

1.51833 

p-level  1 -tailed 

0.48327  p-level  2-tailed 

0.96654 

HO  (5%)? 

accepted 
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Table  17.  T-test  for  Equality  of  Means,  Walking  and  Biking 


T-test  for  Equality  of  Means,  Walking  and  Biking 

Descriptive  Statistics 

VAR 

Sample  size 

Mean 

Standard  Deviation 

Variance 

Walking 

64 

95.57292 

9.28217 

86.15862 

Biking 

64 

84.29036 

9.23304 

85.24912 

t-test  assuming  equal  variances  (homoscedastic) 

Degrees  of  Freedom 

126 

Hypothesized  Mean  Difference 

0. 

Pooled  Variance 

85.70387 

Test  Statistics 

6.89417 

Two-tailed  distribution 

p-level 

2.3245E-10  Critical  Value  (5%) 

1.97897 

One-tailed  distribution 

p-level 

1.16225E-10  Critical  Value  (5%) 

1.65704 

Finally,  a  test  for  differences  in  the  interaction  times  for  obstacles  was 
performed.  After  an  initial  test  for  homogeneity  (Table  18)  of  sample 
variances,  a  T-test  was  performed,  resulting  in  no  significant  difference 
(Figure  19).  In  this  regard,  the  MobileApp  is  different  from  the  WebApp. 
The  MobileApp  performed  similarly  for  both  obstacles,  while  the  WebApp 
did  not.  Conjecture  about  this  dynamic  is  broad,  but  focuses  on  the 
possibility  of  temporal  variations  in  cell  phone  and  local  network 
performance,  which  is  a  critical  factor  for  the  WebApp  but  not  as 
influential  for  the  MobileApp,  which  communicates  less  frequently  with 
our  project  servers,  and  does  not  require  server-based  calculations  or 
feedback. 


Table  18.  F-testfor  Homogeneity  of  Variances:  Obstacle  367  &  Obstacle  0411 


F-Test  Two-Sample  for  Variances 

Descriptive  Statistics _ 

VAR  Obstacle  0411  Obstacle  367 


Sample  size 

Mean 

Variance 

Standard  Deviation 
Mean  Standard  Error 

64 

88.97786 

56.69582 

7.52966 

0.94121 

64 

90.88542 

177.52194 

13.32374 

1.66547 

Summary 

F 

3.1 311 3  F  Critical  value  (5%) 

1.51833 

p-level  1 -tailed 

5.4611  E-6  p-level  2-tailed 

0.00001 

HO  (5%)? 

rejected 
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Table  19.  T-testfor  Equality  of  Means,  Obstacle  367  and  Obstacle  11 


T-test  for  Equality  of  Means,  Obstacle  367  and  Obstacle  0411 

Descriptive  Statistics 

VAR 

Sample  size 

Mean 

Standard  Deviation 

Variance 

Obstacle  367 

64 

90.88542 

13.32374 

177.52194 

Obstacle  0411 

64 

88.97786 

7.52966 

56.69582 

t-test  assuming  unequal  variances  (heteroscedastic) 

Degrees  of  Freedom 

100 

Hypothesized  Mean  Difference 

0. 

Pooled  Variance 

117.10888 

Test  Statistics 

0.99714 

Two-tailed  distribution 

p-level 

0.3211  Critical  Value  (5%) 

1.98397 

One-tailed  distribution 

p-level 

0.16055  Critical  Value  (5%) 

1.66023 

Statistical  Comparisons  of  the  WebApp  and  MobileApp 

A  view  of  the  distribution  of  interactions  times  provided  by  the  Box  Plots 
(Figure  70)  leads  to  a  preliminary  hypothesis  that  there  is  a  significant 
difference  between  the  mean  distances  at  which  an  end-user  receives  an 
obstacle  alert  notification.  As  noted  previously,  the  MobileApp,  due  to  its 
design,  is  more  responsive.  A  statistical  validation  of  this  hypothesis  is 
seen  inTable  20  and  Table  21,  where  a  mildly  significant  F-value  (1.56) 
leads  to  a  T-test  with  accommodations  for  heteroskedasticity.  This  T-test 
(Table  21)  results  in  a  t-test  statistic  of  15.38,  which  is  far  enough  into  the 
distributional  tail  that  the  p-value  is  not  displayed,  other  than  noted  to  be 
o.  The  critical  value  for  this  test  is  1.969.  Clearly,  the  obstacle  notification 
times  for  the  MobileApp  are  delivered  much  closer  to  the  100-foot 
interaction  buffer  than  for  the  WebApp. 
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Box  Plot,  WebApp  and  MobileApp 
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Variable 

Count 

Mean 

Minimum 

Lower  whisker 

Q1 

Median 

Q3 

Upper  whisker 

Maximum 

WebApp 

128 

66.38477 

32.33333 

52.86261 

58.97917 

67.20833 

74.33333 

79.90692 

97.41667 

MobileApp 

128 

89.93164 

64.25 

79.11021 

82.8975 

90.67 

96.105 

100.75307 

115.67 

Figure  70.  Box  Plots  for  Obstacle  Interaction  Distances,  WebApp  (left)  and  MobileApp  (right) 


Table  20.  F-testfor  Homogeneity  of  Variances,  MobileApp  and  WebApp 


F-Test  Two-Sample  for  Variances 


Descriptive  Statistics 


VAR 

MobileApp 

WebApp 

Sample  size 

128 

128 

Mean 

89.93164 

66.38477 

Variance 

117.10343 

182.84866 

Standard  Deviation 

10.82143 

13.52215 

Mean  Standard  Error 

0.95649 

1.1952 

Summary 


F 

1 .56143  F  Critical  value  (5%) 

1.34046 

p-level  1 -tailed 

0.00629  p-level  2-tailed 

0.01257 

HO  (5%)? 

rejected 
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Table  21.  T-test  for  Equality  of  Means,  WebApp  and  MobileApp 


T-test  for  Equality  of  Means,  WebApp  and  MobileApp 

Descriptive  Statistics 

VAR 

Sample  size 

Mean 

Standard  Deviation 

Variance 

WebApp 

128 

66.38477 

13.52215 

182.84866 

MobileApp 

128 

89.93164 

10.82143 

117.10343 

t-test  assuming  unequal  variances  (heteroscedastic) 

Degrees  of  Freedom 

242 

Hypothesized  Mean  Difference 

0. 

Pooled  Variance 

149.97604 

Test  Statistics 

15.38198 

Two-tailed  distribution 

p-level 

0.  Critical  Value  (5%) 

1.96982 

One-tailed  distribution 

p-level 

0.  Critical  Value  (5%) 

1.65117 

Dynamic  Obstacle  Interaction:  Video  and  Demonstration 

Statistical  and  graphical  summaries  of  the  GMU-GcT  dynamic  obstacle 
engagement  study  are  useful.  They  indicate  a  more  responsive  delivery  of 
alerts  with  the  MobileApp,  which  is  due  to  many  of  the  same  issues  noted 
by  Jobe  (2013).  The  WebApp,  with  some  innate  latency,  was  slower  in 
delivering  obstacle  alerts,  which  resulted  in  alert  distances  between  50% 
and  80%  of  the  intended  100  foot  distance.  Useful  accompaniments  to  this 
analysis  are  the  following  videos  (Table  22),  which  demonstrate  the  alert 
functionality  of  the  WebApp  and  MobileApp.  The  videos  contain  side-by- 
side  views  of  the  forward-looking  GoPro,  and  the  respective  screens  of  the 
WebApp  and  the  MobileApp,  and  basic  functionality  of  these  software 
packages  can  be  seen.  The  applications  are  not  commercial  software,  but 
were  coded  to  provide  a  proof-of-concept,  to  provide  dynamic  extensions 
to  the  GMU-GcT,  and  to  test  the  limits  of  current  mobile  software 
development  environments  for  geospatial  applications. 


Table  22.  Obstacle  Interaction  Videos 


Obstacle  367  (westbound) 
Obstacle  367  (eastbound) 
Obstacle  Oil  (westbound) 

Full  videos 


Web  Application  Mobile  Application 


http://geo.gmu.edu/videos/webapp  l.mp4 

http://geo.gmu.edu/videos/mobileapp  l.mp4 

http://geo.gmu.edu/videos/webapp  2. mp4 

http://geo.gmu.edu/videos/mobileapp  2. mp4 

http://geo.gmu.edu/videos/webapp  3. mp4 

http://geo.gmu.edu/videos/mobileapp  3. mp4 

http://geo.gmu.edu/videos/webapp  full. mp4 

http://geo.gmu.edu/videos/mobileapp  full. mp4 
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Mobile  Routing  in  the  GMU-GcT 

General  routing  functionality  in  the  GMU-GcT  was  reviewed  by  Rice  et  al. 
(2014)83  and  Qin  et  al.  (20i5a)84.  Recent  work  has  focused  on  the 
development  of  mobile  routing  routines  in  the  GMU-GcT  with  two 
purposes.  First,  moderators  have  an  interest  in  using  the  MobileApp  to 
explore  the  region,  or  to  purposefully  navigate  toward  and  engage  with 
obstacles  needing  moderation.  In  either  case,  they  are  not  attempting  to 
avoid  engagement  with  obstacles.  This  exploration  functionality  in  the 
MobileApp  allows  the  moderators  to  report  obstacles  using  device  GPS 
location,  attach  photographs,  and  perform  quality  assessments.  Second, 
some  end-users  have  an  interest  in  defining  routes  that  avoid  obstacles, 
stairs,  and  steep  paths,  recognizing  that  this  mode  frequently  results  in 
longer  shortest  cost  paths  (as  presented  in  the  next  chapter).  The  GMU- 
GcT  uses  a  default  mode  titled  ‘Explore’  where  routes  are  not  defined. 
With  a  simple  finger  tap  on  the  route  button  at  the  top  of  the  MobileApp 
(Figure  59,  Figure  60),  end-users  can  select  between  a  standard  shortest 
cost  path  route,  and  an  obstacle-avoiding  route.  In  this  mode,  obstacles 
will  not  be  directly  engaged  because  pedestrian  network  segments 
containing  obstacles  will  not  be  used.  Our  general  analysis  of  routing  is 
contained  in  the  next  chapter  (Chapter  5),  and  views  of  the  ‘Explore’  and 
‘Route’  functionality  are  shown  in  Figure  71  and  Figure  72,  respectively. 
These  examples  of  GMU-GcT  were  generated  with  the  MobileApp  running 
on  an  iPad. 


83  Rice  et  al.,  “Quality  Assessment  and  Accessibility  Applications  of  Crowdsourced  Geospatial  Data:  A 
Report  on  the  Development  and  Extension  of  the  George  Mason  University  Geocrowdsourcing 
Testbed,”  201. 

84  Qin  et  al.,  “Geocrowdsourcing  and  Accessibility  for  Dynamic  Environments.” 
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Figure  71.  Dynamic  Obstacle  Engagement  in  GMU-GcT  MobileApp,  in  "Explore"  mode  (iPad) 
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Figure  72.  Dynamic  obstacle  engagement  in  GMU-GcT  MobileApp,  in  regular  "Route"  mode 

(iPad) 

The  next  chapter  addresses  an  important  extension  of  the  GMU-GcT  in  the 
area  of  routing,  which  is  premised  on  our  past  work  (and  work  presented 
in  this  report)  on  quality  assessment  and  social  moderation.  We  discuss 
how  the  GMU-GcT  provides  an  ability  to  analyze  accessibility  routing  in 
the  local  area.  This  work  builds  on  earlier  work  presented  in  Rice  et  al. 
(2012b,  2013b,  20i4)85>86>87  and  Qin  et  al.  (2015a).88 


85  Rice  et  al.,  “Crowdsourced  Geospatial  Data:  A  Report  on  the  Emerging  Phenomena  of  Crowdsourced 
and  User-Generated  Geospatial  Data.” 

86  Rice  et  al.,  “Crowdsourcing  to  Support  Navigation  for  the  Disabled:  A  Report  on  the  Motivations, 
Design,  Creation  and  Assessment  of  a  Testbed  Environment  for  Accessibility.” 

87  Rice  et  al.,  “Quality  Assessment  and  Accessibility  Applications  of  Crowdsourced  Geospatial  Data:  A 
Report  on  the  Development  and  Extension  of  the  George  Mason  University  Geocrowdsourcing 
Testbed.” 

88  Qin  et  al.,  “Geocrowdsourcing  and  Accessibility  for  Dynamic  Environments.” 
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5  Dynamic  Use  of  the  GMU 

Geocrowdsourcing  Testbed:  Routing 
Analysis 

Obstacle  Avoidance  Routing  in  Pedestrian  Networks 

This  chapter  addresses  an  important  dynamic  use  of  the  GMU-GcT:  the 
ability  to  generate  obstacle-avoidance  routes  across  the  local  pedestrian 
network  using  geocrowdsourcing,  quality  assessment,  and  basic 
information  about  the  accessibility  of  the  pedestrian  network.  To  articulate 
how  this  dynamic  GMU-GcT  functionality  works,  we  present  the 
generation  of  the  current,  underlying  pedestrian  network,  the  basic 
characteristics  of  this  network,  the  geocrowdsourced  obstacles,  and  their 
corresponding  influence  on  routing  results.  As  reviewed  in  Rice  et  al. 
(2013a,  20i3b)89.9o  and  Qin  et  al.  (20isa)91,  capturing  and  documenting 
obstacle  information  on  pedestrian  pathways  can  be  difficult  and  time 
consuming,  depending  on  the  design  of  the  system  and  data  model 
specifications.  Formal  approaches  for  modeling  accessibility  obstacles, 
such  as  those  proposed  by  Laakso  et  al.  (2013)92  and  Chen  et  al.  (2015)93 
are  thorough,  but  not  well  adapted  for  transient  events.  A  crowdsourcing 
approach  with  a  relatively  less  structured,  flexible  workflow  for  collecting 
transient  obstacle  and  infrastructure  data  has  shown  to  be  effective  in  our 
local  area  (Paez  2014,  Rice  2015)94,95.  This  approach  presupposes  an 
effective  quality  assessment  implementation,  where  a  team  of  moderators 
performs  a  quality  check  on  crowdsourced  data.  The  quality  of  this 
moderation  approach  has  been  addressed  in  Chapter  3  of  this  report. 


89  Rice  et  al.,  “Crowdsourcing  Techniques  for  Augmenting  Traditional  Accessibility  Maps  with  Transitory 
Obstacle  Information.” 

90  Rice  et  al.,  “Crowdsourcing  to  Support  Navigation  for  the  Disabled:  A  Report  on  the  Motivations, 
Design,  Creation  and  Assessment  of  a  Testbed  Environment  for  Accessibility.” 

91  Qin  et  al.,  “Geocrowdsourcing  and  Accessibility  for  Dynamic  Environments.” 

92  Mari  Laakso  et  al.,  “An  Information  Model  for  Pedestrian  Routing  and  Navigation  Databases 
Supporting  Universal  Accessibility,”  Cartographica:  The  International  Journal  for  Geographic 
Information  and  Geovisualization  48,  no.  2  (2013):  89-99. 

93  Min  Chen  etal.,  “An  Object-Oriented  Data  Model  Built  for  Blind  Navigation  in  Outdoor  Space,”  Applied 
Geography  60  (2015):  84-94. 

94  Paez,  “Recruitment,  Training,  and  Social  Dynamics  in  Geo-Crowdsourcing  for  Accessibility.” 

95  Rice,  “Validating  VGI  Data  Quality  in  Local  Crowdsourced  Accessibility  Mapping  Applications:  A  George 
Mason  University  Case  Study.” 
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Two  obstacle  types  are  collected  by  the  GMU-GcT:  permanent  obstacles 
and  transient  obstacles.  Permanent  obstacles  include  stairways  or  steps, 
paths  with  a  steep  slope,  narrow  sidewalks,  crosswalks,  curb  cuts,  pathway 
surface  conditions  (such  as  gravel,  cobble  stones,  etc.),  protrusions,  and 
urban  furniture  (such  as  a  trash  cans,  benchs,  or  tables  blocking  a 
pathway).  Transient  obstacles  include  vehicles,  construction  detours, 
construction  barricades,  temporary  fencing  and  barriers  related  to  crowds 
or  special  events.  These  transient  obstacles  are  crowdsourced  and  checked 
by  moderators. 

Pedestrian  Networks 

Based  on  a  review  of  pedestrian  routing  across  several  large  online 
mapping  websites  (Qin  et  al.  20i5a)96  it  is  known  that  many  routing 
services  for  pedestrians  utilize  roadways  instead  of  sidewalks.  Presumably, 
this  is  due  to  the  difficulty  of  assembling  the  underlying  data  for  an 
effective  pedestrian  network,  which  would  —  at  a  minimum  —  include 
sidewalk  centerlines  and  crosswalks.  Figure  73  shows  a  typical  Google 
Maps  example  for  routing  between  two  proximate  buildings  on  the  GMU 
university  campus,  with  routing  on  maps  and  written  directions.  For 
mobility  and  visually  impaired  users,  the  routing  shown  is  not  adequate. 
Sidewalk  centerlines  and  crosswalks,  if  they  exist  and  are  used  in  the 
routing  process,  are  not  sufficient  by  themselves.  Mobility  and  visually 
impaired  users  also  require  information  about  the  presence  of  ramps,  curb 
cuts,  running  and  cross  slope  along  the  pedestrian  network,  sidewalk 
widths,  and  the  accessibility  of  crossing  transition  points.  These  items  are 
difficult  to  collect  and  are  generally  not  included  in  standard  pedestrian 
routes  such  as  those  shown  in  Figure  73. 


96  Qin  et  al.,  “Geocrowdsourcing  and  Accessibility  for  Dynamic  Environments.” 
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Figure  73.  Google  Maps  pedestrian  routing  on  the  GMU  Campus  (September  2014) 

An  exemplar  for  accessible  routing  that  includes  some  accessibility-related 
data  is  the  Accessible  Paths  in  Pinhel  Web  Portal,  where  information  about 
sidewalk  widths,  surface  conditions,  and  running  slope  are  used  to 
generate  an  accessible  route. 97  For  the  local  area,  the  only  pedestrian 
infrastructure  data  that  existed  prior  to  this  project  was  an  incomplete 
ArcGIS  dataset  with  selected  sidewalk  edges,  digitized  by  the  Fairfax 
County  GIS  Team  using  the  Virginia  Base  Mapping  Program  (VBMP) 
Orthoimagery  collected  and  provided  by  the  Virginia  Information 
Technologies  Agency  (VITA)y8  Using  this  preliminary  dataset  as  a  guide, 
along  with  some  incomplete  sidewalk  data  from  the  GMU  campus,  a 
detailed  pedestrian  network  and  supporting  datasets  were  generated  to 
support  this  research,  as  discussed  in  Rice  et  al.  (20i3b)99. 

The  pedestrian  network  generated  for  this  project  (Figure  74)  was  digitized 
from  the  existing  incomplete  sidewalk  datasets  and  VBMP  orthoimagery, 
and  incorporated  into  the  GMU-GcT,  as  noted  in  Rice  et  al.  (2013b)100  and 
Qin  et  al.  (2015a)101.  The  pedestrian  network  encompasses  the  GMU 
campus  and  adjacent  portions  of  Fairfax  County  and  the  City  of  Fairfax. 
The  selected  study  area  was  chosen  in  collaboration  with  City  of  Fairfax 
and  GMU  Campus  planning  and  transportation  agencies  to  represent 
common  origin  and  destination  points  for  commuters,  including  blind, 


97  http://percursos.pinhel.proasolutions.pt/  [accessed  August  23,  2015] 

98  http://www.vita.virginia.gov/isp/default.aspx7ich8412  [accessed  August  23,  2015] 

99  Rice  et  al.,  “Crowdsourcing  to  Support  Navigation  for  the  Disabled:  A  Report  on  the  Motivations, 
Design,  Creation  and  Assessment  of  a  Testbed  Environment  for  Accessibility.” 

100  ibid. 

101  Qin  et  al.,  “Geocrowdsourcing  and  Accessibility  for  Dynamic  Environments.” 
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visually-impaired,  and  mobility-impaired  users  that  walk,  ride  shuttles,  or 
commute  to  campus. 


Figure  74.  A  section  of  the  pedestrian  network  for  the  local  area  (in  yellow). 

The  pedestrian  network  in  the  GMU-GcT  includes  sidewalk  centerlines, 
stairs,  steps,  steep  paths  (with  greater  than  1:12  slope),  crosswalks,  and 
curb  cuts.  The  embedded  pedestrian  network  contains  3,489  network 
segments  covering  an  area  of  5.97  square  miles,  centered  on  the  GMU 
Campus  and  the  City  of  Fairfax. 

Pedestrian  Network  Coverage 

Figure  75  shows  the  roadways  in  the  study  areas  without  adjacent 
sidewalks,  and  Figure  76  shows  an  alternative  representation  with 
pedestrian-inaccessible  areas  of  the  region  masked  with  polygons.  The 
total  length  of  roadways  in  the  study  area  is  187.2  kilometers  and  the  total 
length  of  the  pedestrian  network  (Figure  74),  with  both  sides  of  the 
roadways  included,  is  230.0  kilometers.  The  total  length  of  the  pedestrian- 
inaccessible  roadways  (Figure  75)  is  58.8  kilometers,  or  31.4%  of  the  total 
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roadway  length.  This  number  appears  consistent  with  the  observed 
conditions  in  the  field  study  area. 


Figure  75.  Roads  (in  black)  without  adjacent  sidewalks  (at  left,  with  all  roads,  and  isolated,  at 

right) 


Figure  76.  Pedestrian-inaccessible  regions  masked  with  polygons 
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Obstacles  and  their  Influence  on  the  Accessible  Pedestrian  Network 

Figure  77  and  Figure  78  show  the  distribution  of  stairs,  steep  paths,  and 
crowdsourced  obstacles  in  the  GMU-GcT.  For  the  density  of  buildings  and 
terrain  variability,  the  stairs  and  steep  paths  are  more  heavily 
concentrated  on  the  GMU  campus  and  at  one  notable  section  in  downtown 
City  of  Fairfax. 


Figure  77.  Location  of  Stairs  and  Steep  pathways  (buffered  for  visibility),  with  pedestrian 

network  (left)  and  isolated  (right) 
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Figure  78.  Location  of  Obstacles  in  the  GMU  Geocrowdsourcing  Testbed,  as  of  October  10, 

2014 

Figure  79  and  Figure  80  show  the  segments  of  the  pedestrian  network  that 
were  impacted  by  the  presence  of  stairs,  steep  paths,  and  transient 
obstacles,  and  include  many  lengthy  segments  in  the  central 
transportation  axis  from  the  downtown  City  of  Fairfax  through  the  center 
of  the  George  Mason  University  campus.  The  total  length  of  pedestrian 
network  segments  currently  impacted  by  stairs  and  steep  paths,  and 
transient  obstacles  is  14.1  kilometers,  or  6.1%  of  the  total  length  of  the 
pedestrian  network.  While  this  proportion  seems  rather  small,  the 
distribution  of  impacted  segments  can,  and  does,  have  a  significant 
influence  on  the  accessible  routes,  as  seen  in  the  next  section. 
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Figure  79.  Location  of  pedestrian  network  segments  impacted  by  obstacles,  with  pedestrian 

network  (left)  and  isolated  (right) 


Figure  80.  Location  of  pedestrian  network  segments  impacted  by  Stairs  and  Steep  Paths 
(purple),  and  Obstacles  (red),  with  pedestrian  network  (left)  and  isolated  (right) 


Routing  and  Accessibility  Dynamics 

It  is  possible  for  a  given  route  chosen  by  a  pedestrian  to  begin  and  end 
anywhere  within  the  pedestrian  network.  In  order  to  capture  this  diversity 
of  possible  routes,  this  research  employed  all  network  junctions  as  poten¬ 
tial  start  and  end  points  of  pedestrian  trips.  The  underlying  pedestrian 
network  (Figure  74)  contains  2,772  junctions,  which  generate  7,681,212 
origin-destination  pairs.  The  majority  of  these  pairs  produce  routing  re¬ 
sults  under  normal  conditions,  while  more  than  1/8  origin-destination 
pairs  failed  to  route,  which  is  indicative  of  the  normal  topological  disrup¬ 
tions  of  a  real-world  pedestrian  network,  where  segments  of  the  network 
are  topologically  stranded  from  larger  sections. 
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In  order  to  study  the  general  accessibility  characteristics  of  the  study  area, 
a  routing  analysis  was  constructed  under  three  general  conditions.  The 
first  condition  is  an  initial  unrestricted  condition,  using  every  possible 
pairwise  combination  of  network  junctions.  This  represents  routing  across 
the  entire  pedestrian  network  without  any  accessibility  constraints 
imposed,  and  resulted  in  6,683,094  successful  routes  and  an  average  route 
length  of  1,834.7  meters  (Table  23).  The  second  condition  is  one  in  which 
the  network  is  restricted  by  stairs  and  steep  paths.  This  condition 
represents  the  most  common  general  accessibility  routing  approach  not 
taking  into  account  transient  obstacles.  This  resulted  in  4,381,686 
successful  routes  and  an  average  route  length  of  2,121.23  meters,  or  an 
increase  of  15.62%  from  the  unrestricted  condition  (Table  23).  As  the  most 
restrictive  condition,  the  third  condition  is  one  in  which  the  network  is 
restricted  by  stairs,  steep  paths,  and  transient  obstacles.  This  condition  is 
the  closest  to  reality  for  many  of  the  end-users  interviewed  for  our 
research  (Rice  et  al.  2013a,  2014).  This  resulted  in  3,868,459  successful 
routes  and  an  average  route  length  of  2,215.5  meters,  or  an  increase  of 
20.76%  above  the  unrestricted  condition  (Table  23). 

Table  23.  Summary  Results  of  Routing  Analysis 


No 

restrictions 

Stairs  &  Steep 
Paths  Restricted 

Stairs,  Steep  Paths,  & 
Obstacles  Restricted 

Junctions 

2772 

2665 

2758 

Routes 

6,683,094 

4,381,686 

3,868,459 

Average  route 
length  (m) 

1,834.70 

2,121.23 

2,215.50 

Increase  (%) 

o% 

15.62% 

20.76% 

i 


Three  routing  result  scenarios  are  presented  in  Figure  81,  Figure  82,  and 
Figure  83,  on  the  following  pages.  They  represent  a  range  of  characteristic 
results  for  the  three  scenarios  in  this  study.  They  include  routing  with 
modest  (Figure  81),  substantial  (Figure  82),  and  immense  (Figure  83) 
increases  in  path  length  under  the  three  conditions,  with  accessibility 
restriction  increasing.  Figure  81  shows  an  initial  routing  scenario  from  an 
origin  to  a  destination  1,982  meters  away.  This  route  takes  the  user 
directly  through  the  center  of  the  GMU  campus.  When  stairs  and  steep 
paths  are  restricted,  the  shortest  cost  path  increases  by  4.4%  to  2,070 
meters,  which  represents  a  small  reroute  or  deviation  from  the  shortest 
cost  path.  The  more  restrictive  condition,  where  transient  obstacles  are 
used  to  eliminate  additional  network  segments,  increases  the  length  of  the 
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path  to  2,115  meters,  a  6.7%  increase  over  the  least  restrictive  condition. 
This  represents  many  routing  scenarios  encountered  in  our  system,  where 
routing  around  stairs,  steep  paths,  and  obstacles  requires  a  modest 
increase  in  path  length. 


Figure  81.  Routing  scenario  1:  modest  route  length  increase 

Figure  82  shows  a  route  from  the  study  with  a  substantial  path  increase 
under  restrictive  accessibility  conditions.  The  first  condition  (no 
restrictions)  results  in  a  path  length  between  origin  and  destination  of 
1,319  meters.  Under  the  second  condition  (stairs  and  steep  paths 
restricted),  the  path  length  changes  modestly,  increasing  to  1,432  meters 
or  8.6%.  Under  the  third  condition  (stairs,  steep  paths,  and  transient 
obstacles  restricted)  the  path  length  increases  substantially  to  4,065 
meters,  for  an  increase  of  208.2%.  This  increase  is  due  to  transient 
obstacles  along  a  main  road,  resulting  in  a  long  route  around  an  entire 
neighborhood,  to  arrive  at  a  destination  close  to  the  south  side  of  the  study 
area.  This  substantial  increase  in  the  length  of  an  accessible  path  happens 
when  transient  obstacles  impact  critical  areas  with  poor  sidewalk 
coverage. 
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Figure  82.  Routing  scenario  2:  substantial  route  length  increase 

Figure  83  shows  the  progression  from  normal  conditions  to  restrictive 
accessibility  conditions  in  one  of  the  most  unusual  examples  covered  by 
this  study.  The  figure  shows  a  short  route  from  an  origin  and  destination 
69  meters  apart,  under  no  restrictions.  Imposing  the  second  condition 
(stairs  and  steep  paths  restricted)  yields  an  enormous  increase  in  length, 
to  1,052  meters  or  an  increase  of  1421%.  The  route  is  increased  even 
further  by  the  imposition  of  the  third  condition  (stairs,  steep  paths,  and 
transient  obstacles  restricted),  with  a  final  route  of  1,294  meters,  or  1915% 
above  the  initial  route  length.  Inspection  of  this  scenario  allows  us  to 
conclude  that  an  unusual  combination  of  stairs,  steep  paths,  obstacles,  and 
man-made  features  (buildings)  causes  a  reroute  of  unusual  length.  This 
scenario  is  rare,  and  represents  one  of  the  worst  cases  of  route  length 
increase  in  the  GMU  study  area  under  restrictive  accessibility  conditions. 


Figure  83.  Routing  Scenario  3:  Immense  route  length  increase 


112 


Discussion 

Routing  in  the  GMU  campus  region  is  difficult  due  to  the  overwhelming 
emphasis  on  vehicular  transportation  as  the  preferred  mode  of  transit. 
While  portions  of  the  local  area  are  urbanized  and  walkable  (notably  the 
City  of  Fairfax),  other  portions  of  the  study  area  remain  largely 
inaccessible  due  to  the  lack  of  sidewalks  along  roadways,  lack  of  curb  cuts 
and  connecting  crossing  points,  and  the  presence  of  transient  obstacles. 
Figure  24  shows  that  large  portions  of  the  study  area  are  inaccessible  due 
to  the  lack  of  sidewalks  and  other  infrastructure  such  as  curb  cuts  and 
crosswalks.  Figure  28  shows  that  large  portions  of  the  study  region  with 
sidewalks  are  in  fact  inaccessible  for  many  individuals  because  of  the 
presence  of  stairs,  steep  paths,  and  transient  obstacles  that  block  access  to 
the  pedestrian  network.  These  transient  obstacles  have  a  very  large  impact 
on  the  accessibility  of  the  local  area.  Figure  81,  Figure  82,  and  Figure  83 
demonstrate  some  of  the  characteristic  results  of  imposing  accessibility 
constraints  on  routing. 

The  routing  analysis  demonstrates  that  restricted  routing  (without  using 
stairs  and  steep  paths)  substantially  increases  the  route  length.  Under  this 
restricted  routing  condition,  the  average  path  length  increase  for  the  study 
was  15.62%  above  normal  (unrestricted)  conditions.  For  the  third 
condition  (stairs,  steep  paths,  and  obstacles  restricted)  the  average  path 
length  increased  20.76%  above  normal  (unrestricted)  conditions.  This 
impact  is  felt  directly  by  blind,  visually-impaired,  and  mobility- impaired 
individuals  who  require  accessible  sidewalks,  but  it  is  also  felt  by  senior 
citizens,  families  with  strollers,  and  individuals  with  minor  and  temporary 
mobility  impairments. 

Church  and  Marston  (2003)102  note  that  route  choice  is  highly  individual, 
and  that  traditional  accessibility  measures  do  not  take  into  account  the 
significant  physical  and  mobility  differences  of  individuals.  Their  2003 
study  demonstrated  these  individual  differences  in  four  individually- 
selected  routing  results  between  the  same  origin  and  destination  with 
lengths  varying  from  198  meters  to  610  meters.  This  308%  increase,  due  to 
individual  preferences,  was  reported  to  represent  common  variation 
between  individuals.  Interviews  with  end-users  of  the  GMU-GcT  included 
anecdotal  information  confirming  the  large  individual  variation  in  routing 


102  Richard  L.  Church  and  James  R.  Marston,  “Measuring  Accessibility  for  People  with  a  Disability, 
Geographical  Analysis  35,  no.  1  (2003):  83-96. 


113 


preferences,  due  to  conditions  such  as  tree  cover,  sunlight,  ambient 
temperature,  and  minor  variations  in  cross  slope  (Rice  et  al.  2014,  Figure 
84).  This  individual  variation  in  routing  preference  makes  a  single 
accessible  routing  portal  difficult  to  implement,  unless  extensive  routing 
customization  is  allowed  for  each  user.  Recent  efforts  have  taken 
individual  preferences  into  consideration  in  the  routing  process,  such  as 
the  work  of  Karimi  et  al.  (2014)1Q3  and  the  Pinhel  Accessibility  Platform 
(2014). 


Figure  84.  Individual  route  preference  (green)  and  the  shortest  cost  path  (blue),  from  Rice  et 

al.  2014. 


Summary 

New  applications  involving  distributed  services  (such  as  Thatcher  2013)104 
are  a  developing  theme  in  geocrowdsourcing,  as  are  analysis  performed 
with  volunteered  and  ambient  data.  This  chapter  presents  a  detailed 
routing  analysis  performed  with  geocrowdsourced  obstacle  data  and 
accessibility  infrastructure  data,  but  collected  and  integrated  into  the 
GMU-GcT.  While  the  analysis  here  identified  the  individual  routes 
influenced  by  barriers  to  pedestrian  travel  and  summarized  the  impact  on 
the  network  as  a  whole,  it  has  treated  each  individual  path  as  if  it  were 


103  Hassan  A.  Karimi,  Lei  Zhang,  and  Jessica  G.  Benner,  “Personalized  Accessibility  Map  (PAM):  A  Novel 
Assisted  Wayfinding  Approach  for  People  with  Disabilities,"  Annals  ofGIS  20,  no.  2  (April  3,  2014):  99- 
108,  doi:  10.1080/19475683.2014.904438. 

104  Thatcher,  “From  Volunteered  Geographic  Information  to  Volunteered  Geographic  Services." 


114 


equally  important  from  the  perspective  of  pedestrian  demand.  Further 
research  can  be  envisioned  that  will  enumerate  the  actual,  or  likely,  routes 
to  be  taken  by  the  typical  daily  pedestrian  population  and  as  a  special  case, 
by  the  disabled  population.  Once  routing  dynamics  are  identified,  a  more 
robust  measure  of  the  magnitude  of  the  problems  caused  by  stationary  and 
transient  obstacles  can  be  determined.  Comparisons  of  shortest-cost 
routes  with  actual  routes,  and  the  specific  differences  between  the  two 
would  yield  important  data  about  routing  preference.  Furthermore, 
knowing  the  demand  for  routes,  and  the  set  of  obstacles  causing  variable 
interference  on  those  routes  would  permit  an  investigation  of  which 
obstacles  to  remove  under  limited  funding  conditions.  One  can  imagine  an 
optimization  problem  that  would  direct  the  planning  for  network  update 
and  maintenance.  The  expanding  and  growing  university  and  local  urban 
area  and  concomitant  interest  in  multimodal  transportation  patterns 
makes  an  expansion  of  this  routing  analysis  a  worthwhile  future  activity. 
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6  Conclusions  and  Future  Prospects  for 
Crowdsourced  Geospatial  Data 


The  purpose  of  this  chapter  is  to  reflect  on  the  research  contributions  of 
this  project,  and  the  general  developments  in  geocrowdsourcing.  As  noted 
in  Sui  et  al.  (2013)1Q5,  rapid  developments  in  technology,  including  smart 
phones,  GPS,  distributed  sensor  networks,  and  cloud  computing,  have 
radically  transformed  the  way  geographic  data  are  collected,  stored, 
analyzed,  and  used.  The  former  definition  of  what  could  be  considered 
geospatial  data,  based  on  traditional  cartographic  production  processes 
has  dramatically  changed.  Several  authors  have  captured  and  articulated 
this  dynamic.  In  addition  to  Sui  et  al.  (2013)  and  many  of  the  chapters  in 
their  book,  we  recommend  Elwood  (2010)106  and  Elwood  et  al.  (20i2)1Q7 
for  the  way  that  the  authors  analyze  and  synthesize  technological,  cultural, 
and  social  aspects  of  geocrowdsourcing. 

A  major  concern  cited  by  Goodchild  (2007)108  is  the  quality  and  reliability 
of  volunteered  geographic  information  (VGI).  To  provide  a  context  for 
discussing  quality  assessment  in  geocrowdsourced  data,  Rice  et  al. 
(20i2b)1Q9  summarize  the  National  Map  Accuracy  Standards  (NMAS)110 
from  1947  and  the  more  recent  National  Standard  for  Spatial  Data 
Accuracy  (NSSDA)111  developed  by  the  Federal  Geographic  Data 
Committee  (FGDC)  for  addressing  the  quality  assessment  of  digital 
geospatial  data.  These  general  standards,  and  approaches  for  quality 


105  Sui,  Elwood,  and  Goodchild,  Crowdsourcing  Geographic  Knowledge  Volunteered  Geographic 
Information  (VGI)  in  Theory  and  Practice. 

106  Sarah  Elwood,  “Geographic  Information  Science:  Emerging  Research  on  the  Societal  Implications  of 
the  Geospatial  Web,”  Progress  in  Human  Geography  34,  no.  3  (2010):  349-57. 

107  Sarah  Elwood,  Michael  F.  Goodchild,  and  Daniel  Z.  Sui,  “Researching  Volunteered  Geographic 
Information:  Spatial  Data,  Geographic  Research,  and  New  Social  Practice,”  Annals  of  the  Association 
of  American  Geographers  102,  no.  3  (May  2012):  571-90,  doi:10. 1080/00045608. 2011.595657. 

108  Goodchild,  “Citizens  as  Sensors:  The  World  of  Volunteered  Geography.” 

109  Rice  et  al.,  “Crowdsourced  Geospatial  Data:  A  Report  on  the  Emerging  Phenomena  of  Crowdsourced 
and  User-Generated  Geospatial  Data.” 

110  “United  States  National  Map  Accuracy  Standards”  (U.S.  Bureau  of  the  Budget,  1947), 
http://nationalmap.gov/standards/pdf/NMAS647.PDF. 

111  U.S.  Geological  Survey,  “Geospatial  Positioning  Accuracy  Standards,  Part  3:  National  Standard  for 
Spatial  Data  Accuracy,”  Federal  Geographic  Data  Committee,  August  19,  2008. 
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assessment  from  the  1980s  and  1990s,  were  used  extensively  during  the 
era  when  quality  assessment  and  accuracy  of  digital  GIS  data  was  a 
primary  research  theme.  Notable  publications  from  this  era  include 
Veregin  and  Hunter  (1998)112,  Goodchild  and  Gopal  (i989)u3,  and  Guptill 
and  Morrison  (1995)114.  Many  of  the  recent  research  papers  in  quality 
assessment  for  geocrowdsourced  data  (i.e.,  Haklay  2010,  Girres  and  Touya 
2010)  refer  back  to  these  earlier  traditional  GIS-based  quality  assessment 
works. u5>116  We  wonder  how  far  the  traditional  approaches  for  quality 
assessment  can  be  stretched  to  fit  what  is  emerging  in  geocrowdsourcing 
and  geosocial  media. 

While  it  is  easy  to  see  the  influences  and  sensibility  of  NMAS,  NSSDA,  and 
GIS  quality  assessment  approaches  in  Girres  and  Touya  (20io)u7,  these 
approaches  begin  to  crumble  when  faced  with  unstructured  geosocial 
media  and  aggregations  of  geocrowdsourced  and  linked  data. 
Measurements  related  to  horizontal  positional  accuracy  begin  to  lose 
meaning  when  dealing  with  disparate  collections  of  semi-structured 
geocrowdsourced  data  and  unstructured  geosocial  media. 

Accepting  that  quality  assessment  methods  will  need  to  expand  to 
accommodate  new  sources  of  data  and  new  combinations  of  existing  data, 
seems  to  be  a  common  theme  of  conversation  at  academic  conferences  and 
university  hallways.  One  approach,  discussed  in  Rice  et  al.  (2012b)  is  to 
consider  fitness-for-use  and  risk  analysis  criteria  when  using 
geocrowdsourced  data  or  geosocial  media.  Is  the  new  source  of  data  or 
information  useful?  Can  it  address  the  geographic  question  or  problem? 

If  the  new  source  of  data  is  incorrect,  what  will  the  consequence  be?  Do 
the  generally  low  costs  of  geocrowdsourced  data  merit  consideration  in 
some  less  critical  roles,  for  instance,  in  validating,  improving,  or  updating 
traditional  geospatial  data  sources  (e.g.,  McCartney  et  al.  2015)?118  In 


112  Howard  Veregin  and  Gary  Hunter,  “Data  Quality  Measurement  and  Assessment,”  Educational 
resource,  The  NCGIA  Core  Curriculum  in  GIScience,  (1998), 
http://www.ncgia.ucsb.edu/giscc/units/ulOO/ulOO_f.html. 

113  Michael  F.  Goodchild  and  Sucharita  Gopal,  The  Accuracy  of  Spatial  Databases  (London;  New  York: 
Taylor  &  Francis,  1989). 

114  S.C.  Guptill  and  Morrsion,  J.L.,  Elements  of  Spatial  Data  Quality,  vol.  202  (Elsevier  Science  Limited, 
1995). 

115  Haklay,  “How  Good  Is  Volunteered  Geographical  Information?” 

116  Girres  and  Touya,  “Quality  Assessment  of  the  French  OpenStreetMap  Dataset.” 
ibid. 

n®  McCartney  et  al.,  “Crowdsourcing  The  National  Map.” 
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some  applications  and  settings,  such  as  in  disaster  response  (Zook  et  al. 
20io)u9,  authoritative  data  may  not  exist,  and  there  are  no  other 
reasonable  options. 

The  quality  assessment  approach  taken  in  this  research  (Rice  2013b, 
2014)120’121,  is  similar  to  Girres  and  Touya  in  that  it  adapts  a  GIS-centric 
quality  assessment  approach  to  semi-structured  geocrowdsourced  data. 
This  approach  is  feasible,  yet  there  are  weaknesses.  The  social  moderation 
approach  presented  by  Goodchild  and  Li  (2012)122  and  implemented  in 
this  research  (Rice  20is)123  relies  on  a  team  of  well-trained  moderators, 
whose  judgment  is  accepted  as  a  replacement  for  ground  truth.  A 
contribution  of  this  report  and  a  benefit  of  our  research  is  the  first  large- 
scale  study  of  positional  accuracy  for  social  moderation,  conducted  by  Ms. 
Rebecca  Rice  as  a  part  of  an  MS  Thesis  project  (20is)124.  As  presented  in 
summarized  form  in  Chapter  2,  the  social  moderation  process 
implemented  in  the  GMU-GcT  results  in  positional  accuracies  between 
2.12  and  5.55  meters,  depending  on  whether  the  reported  obstacle  is  more 
similar  to  a  point,  or  polygon.  Rice  (2015)  finds  the  positional  accuracy  for 
the  social  moderation  in  the  GMU-GcT  to  be  equivalent  or  better  than 
similar  positional  assessments  of  geocrowdsourced  data,  as  summarized  in 
Chapter  3  of  this  report. 

An  important  aspect  of  the  success  for  a  geocrowdsourcing  project  is 
whether  or  not  the  project  is  able  to  attract  a  committed  user  base.  While 
applications  such  as  Waze  count  end-users  and  contributors  in  the  tens  of 
millions,  the  GMU-GcT  has  a  user  contributor  pool  counted  in  the  high 
double  digits.  Our  area  of  interests,  geographic  scope,  and  goals  are 
substantially  different.  Throughout  the  duration  of  this  project,  outside 
reviewers  and  collaborators  have  encouraged  us  to  both  radically  simplify, 
and  substantially  ‘gamify’  the  GMU-GcT  in  order  to  attract  contributors. 


119  Matthew  Zook  et  al.,  “Volunteered  Geographic  Information  and  Crowdsourcing  Disaster  Relief:  A  Case 
Study  of  the  Haitian  Earthquake,”  World  Medical  &  Health  Policy  2,  no.  2  (July  21,  2010):  6-32, 
doi:10. 2202/1948-4682. 1069. 

120  Rice  et  al.,  “Crowdsourcing  to  Support  Navigation  for  the  Disabled:  A  Report  on  the  Motivations, 
Design,  Creation  and  Assessment  of  a  Testbed  Environment  for  Accessibility.” 

121  Rice  et  al.,  “Quality  Assessment  and  Accessibility  Applications  of  Crowdsourced  Geospatial  Data:  A 
Report  on  the  Development  and  Extension  of  the  George  Mason  University  Geocrowdsourcing 
Testbed.” 

122  Goodchild  and  Li,  “Assuring  the  Quality  of  Volunteered  Geographic  Information.” 

123  Rice,  “Validating  VGI  Data  Quality  in  Local  Crowdsourced  Accessibility  Mapping  Applications:  A 
George  Mason  University  Case  Study.” 

124  Ibid. 
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They  suggest  that  if  the  contribution  process  is  quick,  easy,  and  fun,  the 
number  of  contributors  will  grow.  Making  a  substantial  change  in  the 
direction  of  the  GMU-GcT  was  difficult,  particularly  with  four  years  of 
scheduled  research  deliverables  that  could  not  otherwise  have  been  met. 
One  suggested  incarnation  of  the  GMU-GcT  is  as  an  Instagram-style 
application  that  focuses  nearly  exclusively  on  the  image.  We  have 
determined  and  noted  that  the  image  is  by  far  the  most  important  aspect 
of  a  geocrowdsourced  contribution  to  the  GMU-GcT,  as  the  image  itself 
and  the  content  of  the  image  header  contain  a  wealth  of  validation 
information.  Preliminary  work  to  extract  information  from  submitted 
images  is  presented  in  Chapter  3,  during  discussion  of  multiposition 
validation.  It  would  be  feasible  to  completely  redesign  the  GMU-GcT  to 
focus  on  image  contribution  with  a  short  twitter-length  message.  The  most 
important  triple  of  geographic  information  (location,  time,  attribute)  could 
be  derived  automatically  from  the  image,  short  message,  and  from  the  web 
submission  or  mobile  submission  process.  Any  extra  information  needed 
could  be  geocrowdsourced  from  the  contributor  community.  While  an 
extensive  quality  assessment  procedure  would  be  missing,  a  greater 
purpose  might  be  served  by  radically  increasing  the  volume  and  coverage 
of  the  GMU-GcT.  The  trade-offs  might  make  this  new  direction 
worthwhile. 

Another  important  conclusion  of  the  research  contributors  to  this  project 
is  a  recognition  of  the  extent  to  which  mobile  devices  have  become  the 
near  complete  focus  for  geocrowdsourcing.  Many  of  the  elements  built  into 
our  original  GMU-GcT  (reviewed  in  Rice  et  al.  20i3b)12s  were  focused  on 
web-based  contributions  done  with  a  desktop  computer.  The  design  of  the 
contribution  tools  changed  over  time  but  preserved  this  desktop  focus,  as 
did  the  moderation  tools,  routing  tools,  and  visualization  tools.  The  recent 
switch  to  mobile  development,  and  the  exploration  of  the  limitations  and 
capabilities  of  mobile  devices  (discussed  in  Chapter  2)  caused  us  to  make 
fundamental  changes  to  the  GMU-GcT  and  the  workflows  for  moderation 
and  quality  assessment.  These  changes  have  been  positive,  as  they  allow 
geocrowdsourcing,  quality  assessment,  and  moderation  to  be  done  from 
the  field  rather  than  from  the  office.  Authors  such  as  Muehlenhaus 
(2013)126,  recognize  this  fundamental  shift  toward  mobile  devices  for  web 
mapping  and  geocrowdsourcing.  Chapter  2  of  this  report  contains  many 


125  Rice  et  al.,  “Crowdsourcing  to  Support  Navigation  for  the  Disabled:  A  Report  on  the  Motivations, 
Design,  Creation  and  Assessment  of  a  Testbed  Environment  for  Accessibility.” 

126  Muehlenhaus,  Web  Cartography:  Map  Design  for  Interactive  and  Mobile  Devices. 
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useful  insights  into  the  basic  capabilities  and  limitations  of  GPS-enabled 
mobile  devices,  and  how  well  they  function  in  dynamic  settings. 

We  welcome  readers  of  this  report  to  send  feedback  and  communicate 
with  us  through  the  project  Principal  Investigator,  Dr.  Matt  Rice.12? 


127  Email:  rice@gmu.edu 
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